Merge canonical main into vterm stage 3
Integrates canonical `main` @2625ec7after PR #137 (tab-width parity) merged. The agreed order was #137 first, this lane second: #137 was approved and FROZEN at5b23e11, and "frozen" is incompatible with "rebase onto the resulting main" — landing it second would have broken its freeze and voided its approval. Integrated by MERGING main into the branch rather than rebasing, matching repo precedent (Merge canonical main into vterm-tui, ... into modeline detection). A rebase would have force-pushed away the review anchors on the two completed review rounds of #135. Main had also moved past this lane's base by #133/#134/#136, so the integration surface was wider than the #135/#137 overlap: src/ semantic_render.rs was a fourth overlapping code file. It auto-merged, as did pmacs-protocol/src/lib.rs. The single code conflict was the pmacs_protocol import list in pmacs-gpu/src/main.rs — TAB_STOP_COLUMNS against the terminal types — resolved as a union. The feared semantic collision did not occur, and this is verified rather than assumed: terminal cell geometry still uses the monospace advance and never TAB_STOP_COLUMNS. pmacs-gpu/src/terminal.rs references neither the constant nor display_width, and terminal_cell_viewport / terminal_run_rect / hit_test_cell derive from mono_advance() and code_line_height() alone. That separation is correct by construction: a terminal's columns come from the child, while tab expansion is a document projection concern. Doc conflicts resolved toward landed state: the tab-width lane moves to "Closed since the last snapshot", the #135/#137 coordination section is kept as a resolved worked example, and the Arc 5 lines in the roadmap and handoff now read "implemented and in review". While resolving, restored a clause main had dropped from the handoff's injection-follow-ups list ("literals, doc-comment code);"), keeping main's strikethrough-and-SHIPPED convention for the modeline entry. Post-integration gates, from a clean tree: cargo fmt --check; strict workspace clippy; pmacs-protocol 17; cargo test --lib 1,768; --features crdt 1,944 (3 ignored each); vterm Stage 1 9/10, Stage 2 4/4, Stage 3 5/7, statusline 7/8, tab-width 2/2 (default/CRDT); M4 121 passed (3 ignored, 1 filtered); required GPU 139; workspace sweep 2,946 passed across 84 suites (19 ignored), one invocation; git diff --check clean.
This commit is contained in:
commit
3c4d969aba
|
|
@ -2583,6 +2583,7 @@ dependencies = [
|
|||
"pmacs-protocol",
|
||||
"pollster",
|
||||
"sys-locale",
|
||||
"unicode-width",
|
||||
"wgpu",
|
||||
"winit",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -14,8 +14,8 @@ backlog.
|
|||
machine-local: `origin` may name this canonical URL, a release mirror,
|
||||
or something else, and therefore has no authority by name alone.
|
||||
- Canonical base at this snapshot:
|
||||
`githubsucks/main` @ `1dd47fc` (modeline detection #132 merged atop Vterm
|
||||
Stage 2 #130; protocol v18).
|
||||
`githubsucks/main` @ `2625ec7` (tab-width parity #137 merged atop
|
||||
locals-query #134 and modeline detection #132; protocol v18 on `main`).
|
||||
- On the transfer source, `origin/main` named a release mirror at
|
||||
`d3fa632` and lagged badly. On the current destination, `origin` names
|
||||
the canonical URL. This difference is why all recovery begins by
|
||||
|
|
@ -49,16 +49,17 @@ git worktree list
|
|||
git status --short --branch
|
||||
```
|
||||
|
||||
The first command must expose `1dd47fc` or a newer intentional main.
|
||||
The first command must expose `2625ec7` or a newer intentional main.
|
||||
If it does not, stop and repair the remote/fetch configuration.
|
||||
|
||||
## Vterm Stage 3 implementation lane
|
||||
|
||||
- Portable branch: `githubsucks/vterm-gpu`
|
||||
- Framing carried as the first commit; implementation follows it.
|
||||
- Base: canonical `main` @ `1dd47fc` (modeline detection #132 atop Vterm
|
||||
Stage 2 #130). Cut from `main`, NOT stacked on `vterm-stage3-framing`,
|
||||
per the framing's §8.
|
||||
- Base: cut from canonical `main` @ `1dd47fc`, NOT stacked on
|
||||
`vterm-stage3-framing`, per the framing's §8. Canonical `main` @
|
||||
`2625ec7` (tab-width parity #137, locals-query #134, modeline handoff
|
||||
#133/#136) is MERGED IN — see the integration entry below.
|
||||
- PR: #135, <https://github.com/levineuwirth/pmacs/pull/135>, open against
|
||||
canonical `main`. Never merge without explicit authorization.
|
||||
- State: criteria 28-37 implemented. Protocol v19 (`SUPPORTED=[6..=19]`):
|
||||
|
|
@ -99,11 +100,31 @@ If it does not, stop and repair the remote/fetch configuration.
|
|||
grid-missing press no longer arms a drag; roadmap/handoff Arc 5 lines
|
||||
corrected. Named deferral: terminal wheel gestures discard scroll
|
||||
magnitude.
|
||||
- Post-round-2 gates: 1,758 default + 1,934 CRDT library tests; required
|
||||
GPU 129; workspace sweep 2,923 across 83 suites; Stage 3 acceptance 5
|
||||
default / 7 CRDT; M4 120; fmt, clippy, diff check clean.
|
||||
- Post-round-2 gates (pre-integration): 1,758 default + 1,934 CRDT library
|
||||
tests; required GPU 129; workspace sweep 2,923 across 83 suites; Stage 3
|
||||
acceptance 5 default / 7 CRDT; M4 120; fmt, clippy, diff check clean.
|
||||
- **Post-integration gates (canonical `main` @ `2625ec7` merged in):**
|
||||
`cargo fmt --check`; strict workspace Clippy; `pmacs-protocol` 17;
|
||||
`cargo test --lib` 1,768; `--features crdt` 1,944 (3 ignored each);
|
||||
vterm Stage 1 9/10, Stage 2 4/4, Stage 3 5/7, statusline 7/8, tab-width
|
||||
2/2 (default/CRDT); M4 121 passed (3 ignored, 1 filtered); required GPU
|
||||
139; workspace sweep 2,946 passed across 84 suites (19 ignored), one
|
||||
invocation; `git diff --check` clean.
|
||||
- Closed caveat: the once-seen required-GPU failure did not reproduce in
|
||||
eight author runs plus five reviewer runs. Treated as environmental.
|
||||
- Canonical-main integration after #137 landed: the agreed order was
|
||||
#137 first (it was approved and FROZEN at `5b23e11`, so it could not
|
||||
absorb a rebase without breaking its freeze), then this lane second.
|
||||
Integrated by MERGING canonical main into the branch, matching repo
|
||||
precedent (`Merge canonical main into vterm-tui`, `… into modeline
|
||||
detection`) rather than a rebase, which would have force-pushed away
|
||||
the review anchors on #135. Main had also moved past this lane's base
|
||||
by #133/#134/#136, so the integration surface was wider than the
|
||||
#135/#137 overlap: `src/semantic_render.rs` was a fourth overlapping
|
||||
code file and auto-merged, as did `pmacs-protocol/src/lib.rs`. The one
|
||||
code conflict was the `pmacs_protocol` import list in
|
||||
`pmacs-gpu/src/main.rs` (`TAB_STOP_COLUMNS` against the terminal
|
||||
types) — resolved as a union.
|
||||
- Next: further user review rounds on the PR.
|
||||
|
||||
Recovery worktree:
|
||||
|
|
@ -115,37 +136,33 @@ git worktree add --track \
|
|||
githubsucks/vterm-gpu
|
||||
```
|
||||
|
||||
## Cross-PR coordination: #135 and #137 overlap
|
||||
|
||||
Two PRs are open against canonical `main` at once and they touch the same
|
||||
files. **Neither lane copies from or merges the other.** PR #137
|
||||
(`tab-width-parity`, "feat(render): unify tab-width projection") is
|
||||
APPROVED and FROZEN at `5b23e11`; this lane must not cherry-pick, merge,
|
||||
or otherwise absorb it.
|
||||
## Cross-PR coordination: #135 and #137 (resolved)
|
||||
|
||||
**Whichever PR lands SECOND rebases onto the canonical resulting `main`
|
||||
and reruns the COMPLETE gate suite** — not a subset, and not the
|
||||
pre-rebase results. The overlap is in the renderer and the protocol
|
||||
crate's export surface, so a clean textual merge does not imply a
|
||||
correct one.
|
||||
**Resolved 2026-07-22: #137 merged first, #135 integrated second.** Kept
|
||||
as the worked example, because the deciding argument is reusable.
|
||||
|
||||
Overlapping paths, and why each one collides:
|
||||
#137 was APPROVED and FROZEN at `5b23e11`. "Frozen" and "rebase onto the
|
||||
resulting main" are mutually exclusive, so the frozen PR had to land
|
||||
first — the alternative would have broken its freeze and voided its
|
||||
approval. Three arguments pointed the same way: the approved PR should
|
||||
not wait on an unapproved one; the PR carrying the protocol byte pins
|
||||
should be the one that integrates, because its own suite is what detects
|
||||
a disturbed discriminant; and the larger, more invasive change should
|
||||
pay the integration cost, since its author has the context to verify the
|
||||
merged result.
|
||||
|
||||
| Path | #135 (vterm stage 3) | #137 (tab width) |
|
||||
| --- | --- | --- |
|
||||
| `pmacs-gpu/src/main.rs` | terminal mode: `State` fields, render-batch swaps, input branches, headless probe | tab-width projection in the same renderer |
|
||||
| `pmacs-protocol/src/lib.rs` | `pub mod terminal` + the terminal re-export block | its own added export |
|
||||
| `Cargo.lock` | `unicode-width` promoted to a workspace dep for `pmacs-protocol` | a `pmacs-gpu` dependency change |
|
||||
| `docs/active-work.md`, `docs/agent-handoff.md` | this lane's entries and §1 snapshot | its own entries |
|
||||
The named overlap (`pmacs-gpu/src/main.rs`, `pmacs-protocol/src/lib.rs`,
|
||||
`Cargo.lock`, both ledger docs) was accurate but incomplete — main had
|
||||
also moved by #133/#134/#136, making `src/semantic_render.rs` a fourth
|
||||
overlapping code file. **Lesson: derive the integration surface from
|
||||
`git diff <base>..main`, not from the other PR's file list.**
|
||||
|
||||
`Cargo.lock` and the docs are mechanical. The two source files are not:
|
||||
both PRs edit the GPU renderer's measurement/paint path, and both widen
|
||||
`pmacs-protocol`'s public surface in the same `pub use` region. After the
|
||||
rebase, re-read the merged region rather than trusting a conflict-free
|
||||
apply — in particular that terminal cell geometry (`cell_viewport`,
|
||||
`hit_test_cell`, `terminal_run_rect`) still uses the monospace advance
|
||||
and is not routed through any new tab-expansion path, since terminal
|
||||
columns come from the child and never from document projection.
|
||||
The feared semantic collision did not occur. Terminal cell geometry
|
||||
still uses the monospace advance and is not routed through
|
||||
`TAB_STOP_COLUMNS`: terminal columns come from the child, and tab
|
||||
expansion is a DOCUMENT projection concern. That is verified rather than
|
||||
assumed — see the integration gates on the lane above.
|
||||
|
||||
## Vterm Stage 3 framing lane (superseded)
|
||||
|
||||
|
|
@ -190,6 +207,16 @@ git worktree add --track \
|
|||
|
||||
## Closed since the last snapshot
|
||||
|
||||
- **Tab-width rendering parity — MERGED as #137** (`main` @ `2625ec7`,
|
||||
2026-07-22). One fixed 8-column `TAB_STOP_COLUMNS` in `pmacs-protocol`
|
||||
now drives core/TUI columns, GPU code projection, and minimap width;
|
||||
source bytes and protocol ranges are unchanged. Its lane and worktree
|
||||
are done. This closes the long-standing "tab width is a rendering-parity
|
||||
bug, NOT a config gap" deferral recorded in `docs/agent-handoff.md` §5.
|
||||
- **Locals-query processing — MERGED as #134** (with handoff #136), and
|
||||
**modeline detection handoff #133**. Both landed between this lane's
|
||||
base and its canonical-main integration.
|
||||
|
||||
- **Config registry — MERGED as #127** (`main` @ `2e37c04`). Its lane
|
||||
(`config-registry`, worktree `../pmacs-config-registry`) is done; the
|
||||
branch is kept but carries nothing unmerged. Durable substrate facts
|
||||
|
|
|
|||
|
|
@ -1,11 +1,12 @@
|
|||
# Agent handoff — cross-machine continuity
|
||||
|
||||
**Last updated: 2026-07-22, after Vterm Stage 3 (protocol v19, GPU
|
||||
terminal) was implemented on `vterm-gpu`. Vterm Stage 2 landed as PR #130 and
|
||||
modeline detection landed as #132. Mode system wiring (#129), config registry
|
||||
(#127), Vterm Stage 1 terminal core (#126), and completed Themes Arc 4
|
||||
(#120/#124/#125) are also on `main`. Stage 3 is implemented on `vterm-gpu`
|
||||
and awaits review; see `docs/active-work.md`.**
|
||||
**Last updated: 2026-07-22, after tab-width rendering parity (#137) and
|
||||
locals-query processing (#134) landed on `main`, and canonical `main` was
|
||||
merged into the Vterm Stage 3 lane. Vterm Stage 3 (protocol v19, GPU
|
||||
terminal) is implemented on `vterm-gpu` and in review as PR #135; see
|
||||
`docs/active-work.md`. Vterm Stage 2 (#130), modeline detection (#132),
|
||||
mode system wiring (#129), config registry (#127), Vterm Stage 1 (#126),
|
||||
and completed Themes Arc 4 (#120/#124/#125) are on `main`.**
|
||||
This file is the
|
||||
bridge between development machines. If you are an agent reading
|
||||
this on a fresh clone: this document plus the `docs/*-framing.md`
|
||||
|
|
@ -19,7 +20,7 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
|
||||
## 1. Where the project stands (2026-07-22)
|
||||
|
||||
- `main` @ `1dd47fc` (modeline detection #132 atop Vterm Stage 2 #130),
|
||||
- `main` @ `2625ec7` (tab-width parity #137 atop locals-query #134),
|
||||
protocol **v18** on `main`, **v19** on `vterm-gpu`
|
||||
(`SUPPORTED=[6..=19]`; v16 = `ThemeFacts`, v17 =
|
||||
`FontFacts`, v18 = `StatuslineSegments`).
|
||||
|
|
@ -83,6 +84,21 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
- The built-in `mode` statusline provider reads `ctx.buffer`, so passive
|
||||
splits render their own mode. Real-daemon acceptance covers all ten framing
|
||||
criteria across both Lua backends and Linux/macOS CI.
|
||||
- **Modeline language detection LANDED — #132**
|
||||
(`docs/modeline-detection-framing.md` rev 2; merge `1dd47fc`). Fresh loads
|
||||
scan bounded Emacs `-*- mode: ... -*-` and Vim/Vi `ft=` / `filetype=`
|
||||
modelines without evaluating file content, normalize common editor aliases,
|
||||
and give explicit modelines precedence over inferred language.
|
||||
- `builtin/runtime/syntax.lua` owns one per-buffer fresh-load decision:
|
||||
modeline → bundled grammar extension → LSP filetype extension → exact
|
||||
filename → shebang. Syntax, initial major mode, pairing, comments, and LSP
|
||||
all reuse that pin; LSP retains its independent backing-path guard.
|
||||
- Editing a modeline or shebang does not switch an attached parser or make
|
||||
language-aware consumers diverge. Close/reopen re-evaluates changed file
|
||||
metadata. Explicit post-load major-mode overrides remain independent.
|
||||
- Bounded valid unknown names remain passive major modes without starting an
|
||||
unavailable parser or server. All thirteen framing criteria are covered on
|
||||
LuaJIT and Lua 5.4. No Rust or protocol surface changed; protocol stays v18.
|
||||
- **Syntax-highlight / language-detection side-quest (#114–#118)
|
||||
LANDED** — a one-shot arc built in sibling worktrees off main while
|
||||
the user's themes lane (`theme-faces`) ran concurrently in the shared
|
||||
|
|
@ -92,25 +108,29 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
the ABI-current `tree-sitter-containerfile`, NOT the dead
|
||||
`tree-sitter-dockerfile` which pins `tree-sitter ^0.20`), make,
|
||||
cmake, python, go, javascript (+jsx), typescript (+tsx), toml, zig.
|
||||
- **Detection chain** (`resolve_active_language` in syntax.lua /
|
||||
`buffer_language` in lsp.lua): extension → LSP filetype map →
|
||||
filename → shebang. New user-extensible Lua surfaces
|
||||
`pmacs.parse.shebangs` / `.filenames` and
|
||||
`pmacs.parse.language_from_shebang` / `language_from_filename`.
|
||||
Grammar name MUST equal the `pmacs.lsp.config.<name>` key (grammar
|
||||
detection wins over the filetype map, so it fixes the LSP id too).
|
||||
A buffer keeps its first-attached grammar across edits/switches (no
|
||||
re-sniff of a since-edited shebang).
|
||||
- **Detection chain and pin** (`builtin/runtime/syntax.lua`): modeline →
|
||||
grammar extension → LSP filetype map → filename → shebang. User-extensible
|
||||
Lua surfaces include `pmacs.parse.modeline_aliases`, `.shebangs`,
|
||||
`.filenames`, `language_from_modeline`, `language_from_shebang`,
|
||||
`language_from_filename`, and the pinned `buffer_language`.
|
||||
`builtin/runtime/lsp.lua` delegates to that shared decision after enforcing
|
||||
its backing-path requirement. Grammar name MUST equal the
|
||||
`pmacs.lsp.config.<name>` key. A buffer keeps its pinned language across
|
||||
edits/switches; close/reopen performs a fresh bounded inference.
|
||||
- **LSP configs added**: dockerfile (`docker-langserver --stdio`),
|
||||
cmake (`cmake-language-server`, config via
|
||||
`init_options.buildDirectory="build"` — it does NOT pull
|
||||
`workspace/configuration`). Make has no server.
|
||||
- **Substrate**: `LanguageEntry.highlights_query` is now
|
||||
`&[&'static str]` — fragments joined base-first, for grammars whose
|
||||
bundled highlights are a `; inherits:` delta (cuda over c/cpp; ts
|
||||
over js/jsx). `compute_highlight_spans` FAILS CLOSED on the
|
||||
`#is?`/`#is-not? local` property predicate (no locals processing) —
|
||||
drops those captures so shadowed builtins aren't mis-styled.
|
||||
- **Substrate**: `LanguageEntry.highlights_query` and `.locals_query` are
|
||||
`&[&'static str]` fragments joined base-first (cuda over c/cpp; ts over
|
||||
js/jsx). Since locals-query processing #134, settle compiles the
|
||||
grammar's `LOCALS_QUERY`, resolves Tree-sitter's scope/definition/value/
|
||||
reference conventions into sorted `LocalFacts`, and stores them beside
|
||||
each layer's tree/query. Work runs once per fresh bundle and only when the
|
||||
highlight query asks about `local`; viewport rendering remains bounded.
|
||||
Both TUI and semantic/GPU producers evaluate `#is?`/`#is-not? local`
|
||||
through the shared capture walk. Non-shadowed JS/TS builtins are restored;
|
||||
shadowed definitions/references keep ordinary variable styling.
|
||||
- **Multi-language injections (#122) LANDED** — the direct continuation
|
||||
of the #114–#118 highlight arc; four review rounds, framing
|
||||
`docs/multi-language-injections-framing.md` (Q#IJ1–IJ11). A buffer can
|
||||
|
|
@ -130,7 +150,8 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
worker never touches the `Rc` registry or Lua);
|
||||
`ParseTreeBundle.injection_capped` (the 4096-layer backstop, surfaced
|
||||
once/buffer via `pmacs.error` at settle);
|
||||
`compute_highlight_spans_for(query, tree, source, range)` (per-layer);
|
||||
`compute_highlight_spans_for(query, tree, source, local_facts, range)`
|
||||
(per-layer);
|
||||
the wire `flatten_layer_spans` event-sweep → DISJOINT effective spans
|
||||
(deeper / later-sibling / narrower wins, keyed by `(layer_index,
|
||||
capture_order)`); GPU `spans_from_segments` + `source_color_at` fold.
|
||||
|
|
@ -279,9 +300,11 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
origin. `pmacs-gpu --headless-probe` drives the real attach client
|
||||
without winit (`attach::connect_with_sink`), which is how criterion 37
|
||||
gets one real daemon + real PTY + real wgpu path.
|
||||
- **Stage 2 TUI LANDED ON `main` — #130** (`docs/vterm-framing.md`
|
||||
Revision 7, criteria 15–27). `TerminalViewKey` keys per-frontend/window
|
||||
projection state over one shared process/screen; logical row anchors retain
|
||||
- **Stage 2 TUI LANDED ON `main` — #130** (merge `86fc1bc`;
|
||||
`docs/vterm-framing.md` Revision 7, criteria 15–27). `TerminalViewKey` keys
|
||||
per-frontend/window projection state over one shared process/screen; logical
|
||||
row anchors retain
|
||||
|
||||
scroll/selection through reflow. One authenticated frontend controls at
|
||||
most one session, with atomic replacement and release on
|
||||
focus/switch/kill/detach.
|
||||
|
|
@ -327,15 +350,28 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
8 CRDT; M4 114 passed (3 ignored, 1 filtered); required GPU 109;
|
||||
workspace 2,882 passed across 82 suites (19 ignored, 1 filtered);
|
||||
`git diff --check` clean.
|
||||
- **Stage 3 framing is Revision 8 on `vterm-stage3-framing`**: additive
|
||||
protocol v19 complete frames/events, dual viewport bootstrap, common
|
||||
validation/aggregate limits, authenticated semantic view adapters, and
|
||||
fixed-cell GPU rendering/input/cache behavior. Criteria 28–37 are mapped.
|
||||
First review found no architectural defect; `c72dfea` pins maximal style
|
||||
and cluster-prefix overhead in the measured frame fixture, corrects the
|
||||
GPU clipboard criterion, and distinguishes Arc 5 stage 2 from its internal
|
||||
Stages 1–3. No implementation branch or PR exists pending explicit user
|
||||
approval.
|
||||
- **Tab-width rendering parity LANDED — #137** (merge `2625ec7`;
|
||||
`docs/tab-width-parity-framing.md` rev 2).
|
||||
Source tabs remain one byte while every buffer
|
||||
renderer follows the shared fixed `pmacs_protocol::TAB_STOP_COLUMNS = 8`.
|
||||
- `src/display_width.rs` owns allocation-free Unicode/tab-aware byte-to-column
|
||||
accounting for plain text, syntax, diagnostics, completion anchors,
|
||||
buffer-style overlays, and search washes.
|
||||
- The GPU rich-chunk projection expands source/adornment tabs before
|
||||
cosmic-text shaping and retains first-class source-tab provenance.
|
||||
Carets, hits, selections, peer washes, and diagnostic geometry share the
|
||||
same source/projected boundary rules, including a soft wrap inside one
|
||||
expanded tab.
|
||||
- GPU minimap widths use the same tab/Unicode rule and refresh in the accepted
|
||||
text-edit transaction. No config, wire shape, negotiation, or protocol
|
||||
version changed. Local gates: 1,763 default + 1,939 CRDT + 1,763 Lua 5.4
|
||||
library tests; 2 focused acceptance; M4 121; required GPU 119; workspace
|
||||
2,911 across 83 suites; strict Clippy and diff check clean.
|
||||
- This closes the standing "**tab width is a rendering-parity bug, NOT a
|
||||
config gap**" deferral in §5: one shared constant now drives every
|
||||
renderer. Terminal cells are deliberately OUTSIDE it — a terminal's
|
||||
columns come from the child, so `pmacs-gpu`'s terminal geometry uses
|
||||
the monospace advance and never `TAB_STOP_COLUMNS`.
|
||||
- **PARKED: kill-ring browser + persistence.** Revision 2 framing is
|
||||
preserved on branch `kill-ring-browser`, but its `0efb5cd` scout is stale
|
||||
and must be repeated before implementation. No PR or implementation is
|
||||
|
|
@ -361,6 +397,10 @@ commands, read `docs/active-work.md` immediately after this file.
|
|||
editing/indent/comment items that were config-blocked.
|
||||
- **Mode system wiring COMPLETE (#129)** — major-mode keymaps,
|
||||
introspection, lifecycle initialization, and statusline display shipped.
|
||||
- **Locals-query processing COMPLETE — #134** — grammar locals metadata,
|
||||
lexical resolution, settled per-layer facts, shared TUI/GPU
|
||||
local-predicate filtering, and a registry-wide locals-query invariant
|
||||
shipped without a protocol change.
|
||||
- Remaining ranked arcs: 6 folding, 7 DAP, 8 GPU splits, plus the
|
||||
`.ipynb` arc (its JSON-grammar prerequisite shipped in #123).
|
||||
|
||||
|
|
@ -521,19 +561,14 @@ cannot detect a discriminant shift.
|
|||
in per-session baselines; and any daemon-side reset needs its
|
||||
frontend mirror audited in the same round (the GPU snapshot arm
|
||||
missed search/menu/status the first time).
|
||||
- **Tab width is a rendering-parity bug, NOT a config gap** (scouted at
|
||||
`7bc0c61` while framing #127; still true). There are FIVE tab-width
|
||||
sites across TWO crates with TWO different values: `TAB_WIDTH = 8` in
|
||||
`src/text_view.rs`, `src/highlight.rs`, `src/diag.rs` and
|
||||
`src/completion.rs`, versus `advance_minimap_col` in
|
||||
`pmacs-gpu/src/main.rs` expanding to **4** — and the GPU's main text
|
||||
path expands tabs *not at all* (buffer bytes reach the frontend raw,
|
||||
so a literal `\t` is shaped by the font). `editor.tab-width` is
|
||||
therefore the obvious-looking first config adopter and is not one:
|
||||
defining the setting cannot make the GPU honor it. Doing it properly
|
||||
needs frontend tab expansion plus a wire-or-frontend-local decision.
|
||||
Deferred from #127 on exactly these grounds; don't re-plan it as a
|
||||
config task.
|
||||
- **Tab width is a rendering semantic, NOT a config gap.** The implementation
|
||||
on `tab-width-parity` fixes the width at the TUI's established 8 columns,
|
||||
shares that constant through `pmacs-protocol`, and expands tabs only in each
|
||||
display projection. Defining `editor.tab-width` could not have fixed the GPU:
|
||||
source text and semantic spans stay byte-addressed while cosmic-text needs
|
||||
projected spaces plus an inverse hit/caret map. A future configurable width
|
||||
would require a buffer-effective frontend fact and cache invalidation; do not
|
||||
re-plan it as a scalar config-only change.
|
||||
- **A test that never runs passes.** Two #127 review-round tests passed
|
||||
vacuously at first: `pmacs.editor.save()` is the RAW save, while
|
||||
`buffer.before-save` fires inside the `buffer.save` COMMAND
|
||||
|
|
@ -626,10 +661,8 @@ and Vim `ft=`/`filetype=` parsing, explicit-over-inferred precedence,
|
|||
alias normalization, and shared fresh-load language pinning for
|
||||
syntax/highlight/LSP startup.
|
||||
Highlight/detection (from the #114–#118 side-quest + injections #122):
|
||||
locals-query processing (run each grammar's LOCALS_QUERY so
|
||||
`#is?`/`#is-not? local` is honored instead of the current fail-closed
|
||||
drop — restores `.builtin` styling for non-shadowed console/require
|
||||
etc.); **injection follow-ups now the engine landed (#122)** —
|
||||
~~locals-query processing~~ **SHIPPED #134**; remaining injection follow-ups
|
||||
now that the engine landed (#122) —
|
||||
`injection.combined` (many matches → one shared parse; PHP-in-HTML, some
|
||||
comment schemes), child-tree incrementality + range-scoped layer rebuild
|
||||
(child layers cold-reparse on every settle today), injectable
|
||||
|
|
@ -637,6 +670,8 @@ runtime/Lua-registered languages (v1 resolves only against
|
|||
`BUILTIN_LANGUAGES`), and the next injection *consumers* gated on new
|
||||
grammars — HTML/CSS/GraphQL/SQL (`<script>`/`<style>`, JS/TS template
|
||||
literals, doc-comment code);
|
||||
~~modeline detection as a 5th layer (`-*- mode: … -*-` /
|
||||
`# vim: ft=…`)~~ **SHIPPED #132**;
|
||||
byte-accurate multibyte cursor placement in `move_active_cursor_to`
|
||||
(still steps one codepoint per LSP byte column). A full Jupyter `.ipynb`
|
||||
setup (reader → editable → kernel execution) now has its JSON grammar
|
||||
|
|
|
|||
|
|
@ -0,0 +1,307 @@
|
|||
# Locals-query processing - syntax-highlight completion
|
||||
|
||||
**Status:** Revision 1 implemented and landed as PR #134 on 2026-07-22
|
||||
(`8cbb9f4`). Protocol remains v18.
|
||||
|
||||
**Base:** `githubsucks/main` at `8bd8298` (mode system #129, Vterm Stage 2
|
||||
#130, modeline detection #132, and landed-state handoff #133). Protocol remains
|
||||
v18.
|
||||
|
||||
## Problem
|
||||
|
||||
pmacs runs each grammar's `highlights.scm` directly through a
|
||||
`tree_sitter::QueryCursor`. That cursor evaluates text predicates such as
|
||||
`#eq?`, `#match?`, and `#any-of?`, but it does not assign or evaluate semantic
|
||||
properties. In particular, JavaScript's bundled highlight query contains:
|
||||
|
||||
```scheme
|
||||
((identifier) @variable.builtin
|
||||
(#match? @variable.builtin "^(arguments|module|console|window|document)$")
|
||||
(#is-not? local))
|
||||
|
||||
((identifier) @function.builtin
|
||||
(#eq? @function.builtin "require")
|
||||
(#is-not? local))
|
||||
```
|
||||
|
||||
A correct highlighter must first run the grammar's `LOCALS_QUERY`, resolve
|
||||
lexical definitions and references, and then apply `#is? local` /
|
||||
`#is-not? local` while selecting highlight captures. Without those facts,
|
||||
a local `console` or `require` can be styled as a builtin.
|
||||
|
||||
The current implementation fails closed by dropping every highlight pattern
|
||||
that carries a `local` property predicate. That prevents the false-positive
|
||||
shadowed-builtin style, but it also drops legitimate builtin highlighting for
|
||||
non-shadowed `console`, `window`, `require`, and peers. This is the first
|
||||
remaining item in the side-quest north-star list at
|
||||
`docs/side-quest-backlog.md:244-249`.
|
||||
|
||||
## Goal
|
||||
|
||||
Run the bundled locals query as part of each settled syntax layer, retain a
|
||||
compact lexical-local map, and use that map to evaluate `#is? local` and
|
||||
`#is-not? local` in both syntax-highlight producers:
|
||||
|
||||
1. `highlight::SyntaxHighlightView` (TUI / overlay path), and
|
||||
2. `semantic_render::scoped_style_spans` (semantic/GPU path).
|
||||
|
||||
A shadowed builtin must retain its ordinary fallback capture but lose the
|
||||
builtin refinement. A non-shadowed builtin must regain the builtin capture.
|
||||
The result must stay correct for nested scopes, TypeScript's inherited locals
|
||||
query, injected-language layers, viewport-limited rendering, and edits that
|
||||
settle a new parse bundle.
|
||||
|
||||
## Scope
|
||||
|
||||
### In
|
||||
|
||||
- Add bundled `locals.scm` fragments to syntax grammar metadata.
|
||||
- Compose inherited locals fragments base-first, exactly as highlight fragments
|
||||
already compose.
|
||||
- Implement Tree-sitter's local-scope conventions:
|
||||
- `@local.scope`
|
||||
- `@local.definition`
|
||||
- `@local.definition-value`
|
||||
- `@local.reference`
|
||||
- `#set! local.scope-inherits false`
|
||||
- Evaluate positive and negative `local` property predicates, including the
|
||||
optional capture-qualified form accepted by Tree-sitter's query parser.
|
||||
- Cache local facts on each settled parse layer.
|
||||
- Feed those facts to both highlight producers.
|
||||
- Cover lexical behavior and end-to-end rendering under both supported Lua
|
||||
backends.
|
||||
|
||||
### Out
|
||||
|
||||
- No new grammar, parser, or language-detection behavior.
|
||||
- No Lua API, command, config key, theme key, protocol field, or frontend-only
|
||||
state.
|
||||
- No LSP semantic-token changes.
|
||||
- No user-defined query registration surface.
|
||||
- No cross-language name resolution between a host layer and an injected
|
||||
child layer.
|
||||
- No Tree-sitter-highlight `reference_highlight` propagation from a local
|
||||
definition's syntactic capture to all references. The target is property
|
||||
predicate correctness. Adding propagation would change ordinary variable
|
||||
styling beyond the reported builtin bug and needs separate framing.
|
||||
- No general engine for arbitrary `#is?` property keys. This side quest owns
|
||||
the Tree-sitter-defined boolean `local` property only; other property
|
||||
predicates retain their current behavior.
|
||||
|
||||
## Existing contracts to preserve
|
||||
|
||||
1. **Layering:** every injection `Layer` owns its grammar tree and highlight
|
||||
query. Deeper layers and later same-depth siblings retain their existing
|
||||
precedence.
|
||||
2. **Viewport work:** the semantic producer's highlight capture walk remains
|
||||
restricted with `QueryCursor::set_byte_range`; rendering a frame must not
|
||||
launch a whole-file locals walk.
|
||||
3. **Settle freshness:** a new parse bundle replaces the old bundle
|
||||
atomically. Local facts must travel with the same bundle, never in a side
|
||||
cache that can pair old scopes with a new tree.
|
||||
4. **Incremental edits:** a completed reparse produces new local facts before
|
||||
the bundle becomes visible. Until settle, producers continue to use the
|
||||
previous internally consistent bundle.
|
||||
5. **Query inheritance:** fragments are newline-joined base-first. Bare
|
||||
concatenation can extend a trailing Scheme comment and is forbidden.
|
||||
6. **Fallback styling:** suppressing a builtin refinement must not suppress
|
||||
an independent ordinary-variable capture for the same identifier.
|
||||
7. **No protocol change:** all work is internal render state.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Q#LQ1 - Grammar metadata carries locals fragments
|
||||
|
||||
`LanguageEntry` gains `locals_query: &'static [&'static str]`, parallel to
|
||||
`highlights_query` and `injections_query`.
|
||||
|
||||
The bundled crates currently exposing `LOCALS_QUERY` are wired as follows:
|
||||
|
||||
| pmacs language | effective locals fragments |
|
||||
| --- | --- |
|
||||
| `lua` | `tree_sitter_lua::LOCALS_QUERY` |
|
||||
| `javascript` | `tree_sitter_javascript::LOCALS_QUERY` |
|
||||
| `javascriptreact` | `tree_sitter_javascript::LOCALS_QUERY` |
|
||||
| `typescript` | JavaScript locals, then TypeScript locals |
|
||||
| `typescriptreact` | JavaScript locals, then TypeScript locals |
|
||||
|
||||
All other entries use an empty slice. TypeScript's locals query is a small
|
||||
parameter-definition delta, so compiling it alone would omit JavaScript's
|
||||
scopes, declarations, and references. JSX uses the JavaScript grammar and
|
||||
therefore the JavaScript locals query. TSX uses the TypeScript crate's TSX
|
||||
language with the same base-plus-delta locals composition.
|
||||
|
||||
`SyntaxRegistry` lazily compiles and caches one effective locals query per
|
||||
language, including cached failure/no-query results, matching the existing
|
||||
highlight-query policy.
|
||||
|
||||
### Q#LQ2 - Local facts follow Tree-sitter lexical semantics
|
||||
|
||||
A locals capture walk maintains a stack of scopes. The stack begins with one
|
||||
non-inheriting root scope covering the layer. Captures are processed in query
|
||||
order and source order:
|
||||
|
||||
1. `@local.scope` pushes a scope covering that node. It inherits outer
|
||||
definitions unless its pattern sets `local.scope-inherits` to `false`.
|
||||
2. `@local.definition` records the identifier in the innermost scope and marks
|
||||
that identifier range local.
|
||||
3. A sibling `@local.definition-value` capture records the initializer/value
|
||||
range. That new definition is not visible while resolving references inside
|
||||
its value, so an outer definition with the same name can still win there.
|
||||
4. `@local.reference` searches definitions newest-first, then scopes
|
||||
innermost-first. Search stops at a non-inheriting scope. A resolved
|
||||
reference range is marked local; an unresolved reference remains non-local.
|
||||
5. Scopes whose end precedes the next capture are popped.
|
||||
|
||||
Definition names are compared as borrowed source byte slices. No identifier
|
||||
strings are allocated. Invalid or out-of-bounds ranges do not produce local
|
||||
facts.
|
||||
|
||||
The result is an opaque `LocalFacts` value containing sorted, deduplicated
|
||||
`(start_byte, end_byte)` ranges. Highlight predicate checks use binary search;
|
||||
they do not hash, copy source text, or rebuild scope state.
|
||||
|
||||
### Q#LQ3 - Predicate evaluation is per emitted capture
|
||||
|
||||
Before emitting a highlight capture, inspect
|
||||
`Query::property_predicates(pattern_index)`:
|
||||
|
||||
- `#is? local` passes only when the selected node is in `LocalFacts`.
|
||||
- `#is-not? local` passes only when the selected node is not in `LocalFacts`.
|
||||
- If the property names a capture, test that capture's node.
|
||||
- If it does not name a capture, test the highlight capture currently being
|
||||
considered, matching Tree-sitter-highlight's per-node behavior.
|
||||
- Multiple `local` predicates on one pattern are conjunctive.
|
||||
- A missing or failed locals query yields no local ranges: negative predicates
|
||||
pass and positive predicates fail. This preserves useful non-local
|
||||
highlighting without inventing local classifications.
|
||||
- Predicates with keys other than `local` remain ignored, preserving the
|
||||
current query engine's scope.
|
||||
|
||||
Text predicates remain the query cursor's responsibility. Property settings
|
||||
such as `local.scope-inherits` are read only by local analysis; they are not
|
||||
mistaken for highlight predicates.
|
||||
|
||||
### Q#LQ4 - Facts are computed once at settle and owned by the layer
|
||||
|
||||
`SyntaxRegistry::resolve_layer_queries` remains the main-thread Stage 2 handoff.
|
||||
For each raw worker layer it:
|
||||
|
||||
1. resolves the cached highlight query;
|
||||
2. checks whether that query contains any `local` property predicate;
|
||||
3. only when needed, resolves the cached locals query and walks the layer tree;
|
||||
4. stores `Arc<LocalFacts>` beside the layer's tree and highlight query.
|
||||
|
||||
This is a single whole-layer analysis per completed parse, not per render.
|
||||
Languages whose highlights never ask about `local` pay only the cheap predicate
|
||||
scan and carry no facts. This includes Lua today even though its locals query
|
||||
is correctly registered for future local-sensitive highlight patterns.
|
||||
|
||||
Putting facts on `Layer` makes the consistency invariant structural:
|
||||
|
||||
```text
|
||||
settled Layer = tree + grammar + highlight query + local facts
|
||||
```
|
||||
|
||||
A producer cannot accidentally retrieve facts for another buffer revision.
|
||||
Child injection layers compute facts against their own tree and grammar; local
|
||||
bindings never cross layer boundaries. A deliberately combined injection
|
||||
layer shares one tree and therefore one lexical environment, matching its
|
||||
combined parse semantics.
|
||||
|
||||
### Q#LQ5 - Both producers share one capture-selection function
|
||||
|
||||
`compute_highlight_spans_for` accepts the layer's optional local facts and
|
||||
owns predicate evaluation. The whole-buffer overlay path and the
|
||||
viewport-limited semantic path both call this function. There is no second
|
||||
predicate implementation in a frontend.
|
||||
|
||||
`compute_highlight_spans_in_range` passes the root layer's facts. Injection
|
||||
producers pass each layer's facts. Existing wider-first ordering and
|
||||
cross-layer merge precedence remain unchanged after captures are selected.
|
||||
|
||||
### Q#LQ6 - Performance boundary
|
||||
|
||||
The locals-query capture walk is linear for one settled layer. Like upstream,
|
||||
resolving references scans visible definitions newest-first, so the worst case
|
||||
is $O(\text{captures} + \text{references} \times \text{definitions})$.
|
||||
It runs only for a language whose highlight query actually contains a `local`
|
||||
predicate, and only once when a fresh parse bundle settles.
|
||||
|
||||
The render hot paths remain:
|
||||
|
||||
- TUI: cached whole-layer highlight spans, rebuilt only when the bundle pointer
|
||||
changes;
|
||||
- semantic/GPU: viewport-bounded highlight query plus binary-search local
|
||||
checks.
|
||||
|
||||
No frame performs a whole-file locals query. Memory is two `u32` offsets per
|
||||
local definition/resolved reference plus vector capacity; ranges are sorted
|
||||
and deduplicated before storage.
|
||||
|
||||
Incremental locals invalidation is intentionally bundle-granular. A local edit
|
||||
can alter all later name resolution in a scope, so attempting to splice only
|
||||
changed ranges without a scope dependency graph risks stale classifications.
|
||||
The parse itself remains incremental; this bounded lexical pass is the boring,
|
||||
correct cutover.
|
||||
|
||||
## Data flow
|
||||
|
||||
```text
|
||||
worker parse
|
||||
-> raw ParseTreeBundle { Layer { tree, language, no queries/facts } }
|
||||
-> main-thread resolve_layer_queries
|
||||
-> cached highlights query
|
||||
-> cached locals query (only if highlights uses local predicates)
|
||||
-> lexical capture walk -> sorted LocalFacts
|
||||
-> settled ParseTreeBundle
|
||||
-> TUI SyntaxHighlightView cache rebuild
|
||||
-> semantic/GPU viewport capture walk
|
||||
-> shared local predicate filter
|
||||
-> existing span ordering/merge/theme lookup
|
||||
```
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
1. **Non-shadowed builtin restored:** JavaScript `console`, `window`, or
|
||||
`require` with no matching lexical definition emits its bundled
|
||||
`*.builtin` capture.
|
||||
2. **Shadowed builtin suppressed:** a parameter or local declaration named
|
||||
`console`/`require` and references resolved to it do not emit a builtin
|
||||
capture; their ordinary variable captures remain.
|
||||
3. **Scope correctness:** shadowing is confined to its lexical scope. A builtin
|
||||
before/after the scope remains builtin, while the definition and references
|
||||
inside are local.
|
||||
4. **Positive predicate:** a focused custom highlight query using `#is? local`
|
||||
emits resolved definitions/references and rejects an unresolved identifier.
|
||||
5. **TypeScript inheritance:** parameter shadowing in TypeScript and TSX uses
|
||||
the JavaScript base locals query plus the TypeScript delta; query compilation
|
||||
and classification succeed for both grammars.
|
||||
6. **End-to-end render:** opening a JavaScript buffer through the shipped
|
||||
runtime and applying a theme that styles only `variable.builtin` renders an
|
||||
unshadowed builtin with that style and a shadowed occurrence without it.
|
||||
7. **Edit freshness:** after changing a shadowing identifier and settling the
|
||||
new parse, the next render reflects the new local/non-local classification;
|
||||
no stale local facts survive.
|
||||
8. **Producer coverage:** both the overlay and semantic/GPU callsites pass the
|
||||
corresponding layer facts to the shared capture walk; injected layers keep
|
||||
their own facts.
|
||||
9. **Backend parity:** the focused acceptance test passes under default Luau
|
||||
and `--no-default-features --features lua54`.
|
||||
10. **No regressions:** formatting, Clippy, default library tests, CRDT library
|
||||
tests, M4 acceptance (excluding the machine-broken basedpyright case),
|
||||
required GPU tests, workspace sweep, and `git diff --check` pass.
|
||||
|
||||
## Expected files
|
||||
|
||||
- `src/syntax.rs` - grammar metadata, locals-query cache, lexical analysis,
|
||||
layer facts, predicate selection, unit coverage.
|
||||
- `src/highlight.rs` - pass per-layer facts to the overlay producer.
|
||||
- `src/semantic_render.rs` - pass per-layer facts to the viewport producer.
|
||||
- `tests/m4_acceptance.rs` - end-to-end local/non-local rendering and edit
|
||||
freshness.
|
||||
- `docs/agent-handoff.md` and `docs/active-work.md` - updated only after the
|
||||
implementation is proven and published according to their protocols.
|
||||
|
||||
No other runtime, Lua, frontend, protocol, or theme file should need a behavior
|
||||
change.
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
# Modeline language detection — side quest
|
||||
|
||||
**Status:** Revision 2, approved for implementation by the user on 2026-07-22.
|
||||
No implementation was present at approval.
|
||||
**Status:** Revision 2 implemented and landed as PR #132 on 2026-07-22
|
||||
(`1dd47fc`). All thirteen acceptance criteria shipped without a protocol change.
|
||||
**Base:** `githubsucks/main` at `d5d9b9c`; protocol v18.
|
||||
|
||||
## Problem
|
||||
|
|
|
|||
|
|
@ -95,8 +95,10 @@ protocol-v18 `StatuslineSegments`.
|
|||
- **Vterm Stage 3 protocol/GPU is implemented and in review (PR #135)**:
|
||||
additive protocol v19 complete frames/events, an aggregate glyph-byte bound
|
||||
under the unchanged transport cap, dual viewport bootstrap, authenticated
|
||||
semantic routing, and native fixed-cell GPU terminal rendering. Landing it
|
||||
closes Arc 5's terminal stage.
|
||||
semantic routing, and native fixed-cell GPU terminal rendering. The 16 MiB
|
||||
transport cap is preserved: the measured legal-worst frame encodes to
|
||||
13,437,863 bytes under an 8 MiB aggregate glyph bound, never chunked.
|
||||
Landing it closes Arc 5's terminal stage.
|
||||
|
||||
### Arc 6 — Folding (keystone gutter rider)
|
||||
|
||||
|
|
|
|||
|
|
@ -25,10 +25,10 @@ cross-cutting index. Keep it pruned as items ship.
|
|||
|
||||
The direct continuation of the #114–#118 grammar/detection stack.
|
||||
|
||||
- **Locals-query processing** — run each grammar's `LOCALS_QUERY` so
|
||||
`#is?`/`#is-not? local` is honored (restores `.builtin` styling for
|
||||
*non-shadowed* console/require etc.), replacing the current
|
||||
fail-closed drop in `compute_highlight_spans`.
|
||||
- **Locals-query processing — SHIPPED (#134).** Bundled Lua/JavaScript/
|
||||
TypeScript locals queries now drive settled per-layer lexical facts, and
|
||||
both highlight producers honor `#is?`/`#is-not? local`. Non-shadowed
|
||||
builtins regain `.builtin` styling while shadowed names stay ordinary.
|
||||
- **Multi-language injections — SHIPPED (#122).** The load-bearing
|
||||
engine landed: `ParseTreeBundle` holds `Vec<Layer>`, the worker builds
|
||||
child trees off the static grammar table, settle resolves per-layer
|
||||
|
|
@ -40,8 +40,8 @@ The direct continuation of the #114–#118 grammar/detection stack.
|
|||
languages (v1 resolves only against `BUILTIN_LANGUAGES`), and new
|
||||
injection *consumers* gated on grammars — HTML/CSS/GraphQL/SQL
|
||||
(`<script>`/`<style>`, template literals, doc-comment code).
|
||||
- **Modeline detection** — a 5th detection layer (`-*- mode: … -*-`,
|
||||
`# vim: ft=…`) after extension → filetype → filename → shebang.
|
||||
- ~~**Modeline detection**~~ — **SHIPPED as #132.** Bounded Emacs/Vim
|
||||
metadata now precedes extension → filetype → filename → shebang inference.
|
||||
- **JSON + YAML — PR #123 open.** Grammars and LSP configs exist on the
|
||||
feature line; review fixes are preserved on
|
||||
`json-yaml-handoff-2026-07-20`. A real YAML-through-pmacs smoke,
|
||||
|
|
@ -120,14 +120,12 @@ The direct continuation of the #114–#118 grammar/detection stack.
|
|||
language-aware indent, per-language comment padding, and per-project
|
||||
compile commands — the last three are now ordinary work, expressed as
|
||||
a `buffer.after-load` hook calling `set_local`, not blocked work.
|
||||
- **Tab-width rendering parity** — was listed above as a config
|
||||
consequence; it is not. `TAB_WIDTH = 8` appears four times in the
|
||||
daemon (`text_view`, `highlight`, `diag`, `completion`), the GPU
|
||||
minimap's `advance_minimap_col` uses **4**, and the GPU main text path
|
||||
expands tabs *not at all* — raw `\t` reaches glyphon and is shaped by
|
||||
the font. Defining `editor.tab-width` cannot make the GPU honor it;
|
||||
this needs frontend tab expansion plus a wire-or-frontend-local
|
||||
decision. Deferred from #127 on those grounds.
|
||||
- ~~**Tab-width rendering parity**~~ — **IMPLEMENTED, IN REVIEW AS #137.** One fixed
|
||||
8-column constant now drives the core/TUI display-column paths, GPU rich-text
|
||||
projection, and minimap widths. GPU expansion retains source-tab provenance,
|
||||
so caret, hit, selection, and diagnostic geometry remain byte-correct through
|
||||
adornments and soft wraps. Source text and protocol ranges remain raw; this
|
||||
adds no config key or wire change. See `docs/tab-width-parity-framing.md`.
|
||||
- **Real `read_only` buffer flag** on both edit paths — true immutability
|
||||
for panels / REPL / generated buffers.
|
||||
- ~~**Mode system wiring**~~ — **SHIPPED as #129.** Per-buffer major modes
|
||||
|
|
@ -237,18 +235,14 @@ guides (visual, not color).
|
|||
|
||||
## North star (highest-leverage first)
|
||||
|
||||
**The original north-star items and mode-system wiring have now shipped** —
|
||||
**The original north-star items have shipped or reached review** —
|
||||
multi-language injections (#122), the config registry (#127), JSON + YAML
|
||||
(#123), and mode-system wiring (#129). The remaining board:
|
||||
(#123), mode-system wiring (#129), locals queries (#134), and tab-width
|
||||
rendering parity (PR #137, review pending). The remaining board now
|
||||
starts with the broader ranked arcs below rather than another unresolved
|
||||
cross-frontend rendering invariant.
|
||||
|
||||
1. **Locals-query processing** — restores `.builtin` styling for
|
||||
non-shadowed builtins, the last rough edge of the highlight stack.
|
||||
2. **Tab-width rendering parity** — five constants across two crates
|
||||
with two different values, and no tab expansion at all on the GPU
|
||||
main text path. Explicitly NOT a config-registry task; see the entry
|
||||
under "Cross-cutting substrate".
|
||||
|
||||
Beyond those, the cleanest one-shots in the highlight family are
|
||||
**modeline detection** and the HTML/CSS grammars that light up more
|
||||
injection *consumers*; the most-missed editing table-stakes remain
|
||||
**word-kills + `C-SPC` set-mark**.
|
||||
Beyond those, the cleanest remaining one-shots in the highlight family are the
|
||||
HTML/CSS grammars that light up more injection *consumers*; modeline detection
|
||||
shipped in #132. The most-missed editing table-stakes remain **word-kills +
|
||||
`C-SPC` set-mark**.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,428 @@
|
|||
# Tab-width rendering parity - side quest
|
||||
|
||||
**Status:** Revision 2 implemented on `tab-width-parity`; all fifteen
|
||||
acceptance criteria pass locally. PR #137 is open for review.
|
||||
|
||||
**Base:** `githubsucks/main` at `40111dc` (landed-state documentation for
|
||||
locals-query processing #134); protocol v18.
|
||||
|
||||
## Problem
|
||||
|
||||
Pmacs has no single tab-rendering contract:
|
||||
|
||||
- `src/text_view.rs`, `src/highlight.rs`, `src/diag.rs`, and
|
||||
`src/completion.rs` independently hard-code an 8-column tab stop.
|
||||
- `src/overlay.rs` repeats the same 8-column arithmetic as literals.
|
||||
- `pmacs-gpu/src/main.rs::advance_minimap_col` uses 4 columns and counts every
|
||||
non-tab character as one column.
|
||||
- The GPU code buffer sends raw `\t` bytes to cosmic-text. Its visible width is
|
||||
therefore whatever the selected font's tab glyph happens to provide, not a
|
||||
pmacs tab stop.
|
||||
|
||||
The disagreement is observable. The same buffer can place text, syntax faces,
|
||||
diagnostic squiggles, completion popups, selections, carets, and minimap marks
|
||||
at different columns between the TUI and GPU frontends. Merely defining an
|
||||
`editor.tab-width` config key would not fix the GPU: tab expansion, source-byte
|
||||
mapping, styling, and hit testing all happen inside the frontend after the raw
|
||||
semantic frame arrives.
|
||||
|
||||
This is the remaining top-ranked item in `docs/side-quest-backlog.md:123-130`
|
||||
and `:245-248`.
|
||||
|
||||
## Goal
|
||||
|
||||
Define one fixed 8-column tab-stop invariant, make every shipped buffer-text
|
||||
renderer honor it, and preserve byte-addressed editor semantics while the GPU
|
||||
shapes an expanded display projection.
|
||||
|
||||
For a tab beginning at logical display column `c`, its width is
|
||||
|
||||
```text
|
||||
8 - (c mod 8)
|
||||
```
|
||||
|
||||
so a tab at an already aligned column advances a full eight columns. Source
|
||||
text remains byte-for-byte unchanged.
|
||||
|
||||
## Scope
|
||||
|
||||
### In
|
||||
|
||||
- One canonical tab-stop constant shared by the core and GPU crates.
|
||||
- One core display-column utility used by plain text, syntax styling,
|
||||
diagnostics, completion placement, and generic buffer-style overlays.
|
||||
- Tab expansion in the GPU code-buffer projection before cosmic-text shaping.
|
||||
- Correct projected-to-source and source-to-projected mapping for clicks,
|
||||
carets, selections, diagnostic geometry, wrapping, and inline adornments.
|
||||
- GPU minimap width/indent accounting using the same tab and Unicode-width
|
||||
rules as the code view.
|
||||
- Focused regression coverage at tab-stop boundaries, after wide Unicode, and
|
||||
through styled and selected tab bytes.
|
||||
|
||||
### Out
|
||||
|
||||
- A user-configurable `editor.tab-width` setting. This change deliberately
|
||||
chooses the already-shipped TUI behavior, 8, and makes it universal.
|
||||
- Per-buffer or per-language tab widths, indentation policy, soft-tab
|
||||
insertion, tab-to-spaces conversion, or retabbing existing files.
|
||||
- Changing what the Tab key inserts. A literal tab remains one source byte.
|
||||
- Expanding tabs in protocol payloads or mutating `SemanticFrame` byte ranges.
|
||||
- Tabs in statusline, minibuffer, menu, hover panels, or other non-buffer UI
|
||||
strings.
|
||||
- A wire schema or protocol-version change.
|
||||
|
||||
## Ground truth and contracts to preserve
|
||||
|
||||
### Core renderers are byte-addressed but paint in display columns
|
||||
|
||||
`TextView` already expands a tab to spaces at the next multiple of 8 and maps
|
||||
the one source byte to that display interval. Syntax highlighting and
|
||||
diagnostics independently translate byte ranges into display columns.
|
||||
Completion computes its popup anchor from a byte offset. Generic
|
||||
`BufferStyleSpan` overlays compute both ends from the line start. All five
|
||||
paths require the same prefix-width operation; today they implement it
|
||||
separately.
|
||||
|
||||
The current TUI inverse mapping rounds every display column inside an expanded
|
||||
tab forward to the source boundary after the tab. That behavior is observable
|
||||
and remains the cross-frontend rule.
|
||||
|
||||
### GPU code text is already a source-preserving projection
|
||||
|
||||
`projected_rich_chunks` interleaves source text, foreground style spans, and
|
||||
inline adornments. `line_from_chunks` is the only content fed to cosmic-text.
|
||||
`line_chunk_cache` drives source-byte-to-layout-cursor conversion, while
|
||||
`current_hit_runs` and `projected_line_starts` convert cosmic-text hit results
|
||||
back into source bytes. Incremental line reshaping and full slice rebuilds both
|
||||
consume the same chunk construction path.
|
||||
|
||||
Tab expansion belongs at this projection boundary. Expanding the daemon's text
|
||||
would invalidate every protocol byte range; asking cosmic-text to interpret raw
|
||||
tabs would retain font-dependent behavior.
|
||||
|
||||
### GPU geometry currently assumes source bytes equal shaped bytes
|
||||
|
||||
Foreground colors are attached to chunks and therefore naturally survive a
|
||||
projection when the chunk provenance is retained. Background selections,
|
||||
current-line washes, and diagnostic squiggles are different:
|
||||
`push_glyph_extent_rects` currently compares source-relative decoration bytes
|
||||
directly with cosmic-text glyph byte offsets. That equality already needs
|
||||
special handling for adornments and becomes definitively false once one tab
|
||||
byte projects to multiple spaces. The geometry path must use the same
|
||||
source/projection mapping as caret placement and hit testing.
|
||||
|
||||
### The protocol transports raw text and raw byte ranges
|
||||
|
||||
`pmacs-protocol` owns the types shared by the daemon and GPU. `SemanticFrame`
|
||||
continues to carry unmodified text plus byte-addressed spans, decorations, and
|
||||
adornments. A tab-stop constant is a rendering semantic for those existing
|
||||
fields, not a serialized field. Adding it changes neither postcard encoding nor
|
||||
version negotiation.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Q#TW1 - The canonical tab stop is fixed at eight columns
|
||||
|
||||
Add a documented public constant named `TAB_STOP_COLUMNS: u32 = 8` to
|
||||
`pmacs-protocol` and re-export it through the crate root. Both the pmacs core
|
||||
and `pmacs-gpu` consume that constant.
|
||||
|
||||
The shared protocol crate is the narrow existing dependency common to both
|
||||
frontends. A second rendering crate is unjustified, while two frontend-local
|
||||
constants would preserve the drift this work is meant to remove. The constant
|
||||
is normative metadata for interpreting raw text already carried by the
|
||||
semantic protocol; it is not serialized.
|
||||
|
||||
Do not add a config-registry key. A future configurable width would need a
|
||||
buffer-effective value in every semantic frame (or another versioned frontend
|
||||
fact), cache invalidation when it changes, and tests across reconnects. That is
|
||||
a separate feature, not hidden scope in this parity fix.
|
||||
|
||||
This work changes no `PROTOCOL_VERSION`: no message variant, field, encoding,
|
||||
capability, or negotiation rule changes. It adds a compiled rendering invariant
|
||||
for a previously unspecified raw-tab case to the protocol version present on
|
||||
its implementation base.
|
||||
|
||||
### Q#TW2 - One core module owns display-column arithmetic
|
||||
|
||||
Add `src/display_width.rs` and export it from `src/lib.rs`. It owns:
|
||||
|
||||
- `TAB_STOP_COLUMNS` consumption from `pmacs_protocol`;
|
||||
- advancing a logical column by one character, including tabs and
|
||||
`unicode-width` handling;
|
||||
- the width of a valid UTF-8 string from a specified starting column;
|
||||
- the display column at a byte boundary in a line; and
|
||||
- the display-column pair for a half-open byte range.
|
||||
|
||||
Byte helpers clamp to the input length and use the longest valid UTF-8 prefix
|
||||
when a stale/asynchronous range lands inside a code point. They do not allocate.
|
||||
Tabs are always evaluated from the line's logical column zero, not from the
|
||||
viewport edge or a range's start.
|
||||
|
||||
Migrate `text_view`, `highlight`, `diag`, `completion`, and `overlay` to this
|
||||
module. Delete their constants and private copies rather than leaving aliases
|
||||
or wrapper functions. `TextView` may still special-case tab painting, but its
|
||||
pad count comes from the shared column advance.
|
||||
|
||||
### Q#TW3 - GPU expands tabs in the rich-chunk projection
|
||||
|
||||
After source/style/adornment boundaries have produced `RichChunk`s, run one
|
||||
projection pass before either full-slice or per-line shaping. The pass walks
|
||||
chunks and code points in display order while tracking a logical display
|
||||
column:
|
||||
|
||||
- ordinary characters retain their text and provenance and advance by
|
||||
`unicode-width`;
|
||||
- newline resets the logical column to zero;
|
||||
- a source tab becomes `TAB_STOP_COLUMNS - (column % TAB_STOP_COLUMNS)` ASCII
|
||||
spaces carrying explicit provenance for that one source byte;
|
||||
- a tab inside an adornment also becomes spaces but retains the adornment's
|
||||
anchor provenance; and
|
||||
- zero-width characters do not advance the logical column.
|
||||
|
||||
A chunk with no tab is retained rather than copied again. Chunks containing
|
||||
one or more tabs are split only at those tab boundaries. The existing visible
|
||||
slice and per-line caches therefore bound both allocations and work; the
|
||||
frontend never expands the whole file merely to draw one viewport.
|
||||
|
||||
`line_from_chunks`, `build_hit_runs`, incremental line replacement, and the
|
||||
full rebuild all consume the expanded chunks. No alternate shaping path may
|
||||
feed raw buffer tabs to cosmic-text.
|
||||
|
||||
### Q#TW4 - A projected tab run has first-class source provenance
|
||||
|
||||
Extend `ChunkSource` with a source-tab form containing the tab's
|
||||
slice-relative byte offset. The derived `ProjectedRun` then represents three
|
||||
semantics:
|
||||
|
||||
1. source text is byte-linear;
|
||||
2. adornment text snaps to its anchor; and
|
||||
3. all projected spaces for a tab correspond to one source byte.
|
||||
|
||||
Boundary rules are explicit:
|
||||
|
||||
- source offset at the tab byte maps to the first projected space;
|
||||
- source offset immediately after the tab maps after the final projected
|
||||
space;
|
||||
- a projected hit exactly at the tab's leading boundary maps before the tab;
|
||||
- any hit inside its expanded interval maps after the tab, matching
|
||||
`TextView::display_to_pos`; and
|
||||
- a hit at the following projected boundary maps to the following source
|
||||
boundary without crossing an adornment's established left-gravity rule.
|
||||
|
||||
Factor the per-line source-to-projected conversion out of
|
||||
`State::code_byte_to_projected` so caret placement, decoration geometry, and
|
||||
unit tests use the same boundary implementation. Keep projected-to-source in
|
||||
the run map built from those exact chunks. Do not infer positions from counts
|
||||
of spaces after shaping.
|
||||
|
||||
### Q#TW5 - Tab stops use the final visible logical column
|
||||
|
||||
The projection pass counts all visible content before a tab, including wide
|
||||
Unicode and inline-adornment text. This makes the expanded tab end on a visible
|
||||
8-column boundary instead of overlapping or drifting when an inlay hint occurs
|
||||
before it.
|
||||
|
||||
Cosmic-text remains responsible for glyph shaping and pixel geometry. The tab
|
||||
rule controls how many monospace spaces are supplied; it does not replace
|
||||
shaping with manual pixel placement. Code font fallback may vary in pixels,
|
||||
but logical columns remain deterministic.
|
||||
|
||||
Add `unicode-width = "0.2"` as a direct `pmacs-gpu` dependency. Do not reach
|
||||
through another crate's transitive dependency.
|
||||
|
||||
### Q#TW6 - Styles and decorations cover the full projected tab
|
||||
|
||||
Foreground styling is preserved by assigning every expanded source-tab chunk
|
||||
the color of the source chunk that contained the tab. A style span covering
|
||||
`[tab, tab + 1)` therefore colors every projected space; a span ending at the
|
||||
tab colors none of them.
|
||||
|
||||
For background selections and diagnostic squiggles, convert each source-range
|
||||
intersection on a shaped line to projected byte boundaries before comparing it
|
||||
with `LayoutGlyph::{start,end}`. The conversion uses that line's cached chunks
|
||||
and the Q#TW4 boundary rules. Do not rewrite protocol ranges, and do not use
|
||||
source line offsets as if they were projected byte offsets.
|
||||
|
||||
This same conversion covers own selections, peer selections, current-line
|
||||
geometry where applicable, and diagnostic ranges. Gutter diagnostic signs are
|
||||
line-presence indicators and remain source-line based; they need no horizontal
|
||||
projection.
|
||||
|
||||
### Q#TW7 - The minimap uses the same logical-width rule
|
||||
|
||||
Replace the hard-coded 4-column `advance_minimap_col` branch with the shared
|
||||
`TAB_STOP_COLUMNS` value. Ordinary characters advance by `unicode-width`
|
||||
instead of unconditionally by one; zero-width characters advance by zero and
|
||||
wide characters by two.
|
||||
|
||||
The minimap remains a density abstraction rather than shaped text, but its
|
||||
indent and content extents now agree with the code view's logical columns.
|
||||
Clipping and pixel compression are unchanged.
|
||||
|
||||
### Q#TW8 - Projection invalidation follows existing text/chunk invalidation
|
||||
|
||||
Tab width is fixed at compile time, so it introduces no runtime invalidation
|
||||
source. Text edits, style/adornment updates, font changes, resizes, scrolling,
|
||||
and buffer switches already rebuild or replace the affected chunk cache. Tab
|
||||
projection runs inside those existing paths.
|
||||
|
||||
`try_reshape_line` must regenerate the expanded chunks for the edited line;
|
||||
`rebuild_lines_reusing_scroll` may retain an unchanged line and its already
|
||||
expanded cache. `hit_map_dirty` continues to mark when the whole-slice reverse
|
||||
map must be rebuilt. No new generation counter or whole-file cache is needed.
|
||||
|
||||
## Data flow
|
||||
|
||||
```text
|
||||
daemon / TUI core
|
||||
source bytes ───────────────────────────────────────────────┐
|
||||
│ │
|
||||
├─ display_width helpers ──> TUI glyph/style columns │
|
||||
│ │
|
||||
└─ SemanticFrame { raw text, byte ranges } ─────────────┤
|
||||
v
|
||||
pmacs-gpu
|
||||
│
|
||||
style + adornment boundaries ─────────┤
|
||||
v
|
||||
source-rich chunks
|
||||
│
|
||||
expand tabs to spaces
|
||||
+ preserve provenance
|
||||
│
|
||||
┌────────────────────────────────┼─────────────┐
|
||||
v v v
|
||||
cosmic-text hit/caret map decoration map
|
||||
shaping/render ↕ source source → glyph
|
||||
│ │ │
|
||||
└────────────────────────────────┴─────────────┘
|
||||
```
|
||||
|
||||
The invariant is that only the display projection expands a tab. Every editor,
|
||||
protocol, edit, selection, syntax, diagnostic, and adornment coordinate remains
|
||||
a source-byte coordinate.
|
||||
|
||||
## Bets
|
||||
|
||||
1. **Eight is the correct parity target.** It is the established TUI behavior
|
||||
and existing tests already encode it. This work removes divergence rather
|
||||
than introducing a new preference.
|
||||
2. **ASCII spaces are the stable shaping input.** The code font is measured as
|
||||
monospace and spaces participate in wrapping, hit testing, and glyph ranges
|
||||
that the existing GPU architecture already understands.
|
||||
3. **Visible-slice expansion is cheap enough.** The GPU already allocates owned
|
||||
rich chunks for the shaped viewport. Scanning them once and allocating only
|
||||
around actual tabs is below shaping cost and avoids a whole-file projection.
|
||||
4. **Forward rounding inside a tab is acceptable.** It matches the shipped TUI
|
||||
inverse mapping and avoids inventing fractional positions inside one source
|
||||
byte.
|
||||
5. **The fixed semantic constant needs no protocol-version change.** This work
|
||||
changes no message representation or negotiation rule. Existing compatible
|
||||
clients remain decodable but must adopt the documented invariant to obtain
|
||||
visual parity.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
1. **Canonical rule:** one exported `TAB_STOP_COLUMNS = 8` definition is shared
|
||||
by the pmacs core and GPU; no renderer-local tab-width literals remain in
|
||||
the touched buffer-rendering paths.
|
||||
2. **Source preservation:** inserting/opening `"\t"` leaves one tab byte in the
|
||||
buffer, semantic frame, edits, undo history, and saved file. Rendering never
|
||||
replaces source text.
|
||||
3. **TUI boundary behavior:** tabs beginning at columns 0, 7, and 8 end at
|
||||
columns 8, 8, and 16 respectively in plain rendering and position mapping.
|
||||
4. **Core overlay parity:** syntax foreground spans, diagnostic underlines,
|
||||
completion popup anchors, and generic buffer-style spans all resolve the
|
||||
same byte boundary after tabs and wide Unicode to the same display column.
|
||||
5. **GPU shaping input:** code-buffer chunks presented to cosmic-text contain
|
||||
no raw tab from source text or text adornments. The equivalent expanded
|
||||
spaces end at the next logical 8-column boundary.
|
||||
6. **GPU visual geometry:** for `"\tx"`, `"1234567\tx"`, and
|
||||
`"12345678\tx"`, the GPU lays out `x` at logical columns 8, 8, and 16.
|
||||
A case with a width-2 Unicode character before the tab also lands on the
|
||||
mathematically correct stop.
|
||||
7. **Caret mapping:** source carets immediately before and after a tab render at
|
||||
the leading and trailing edges of the expanded interval, including when the
|
||||
line wraps near that interval.
|
||||
8. **Hit testing:** clicking the tab's leading boundary resolves before the tab;
|
||||
clicking within its expanded spaces resolves after it; clicking following
|
||||
text resolves to its original source byte. No click returns a synthetic
|
||||
space offset.
|
||||
9. **Styled tab:** a source foreground span exactly covering a tab colors every
|
||||
expanded space and does not color the following source character.
|
||||
10. **Selected/diagnostic tab:** own and peer selections and diagnostic
|
||||
squiggles whose source range covers a tab span the full projected interval
|
||||
and remain aligned with following text. The GPU layout case includes a soft
|
||||
wrap whose boundary falls inside the expanded tab, proving that one source
|
||||
byte produces correct geometry on both visual lines.
|
||||
11. **Adornment interaction:** an inline text adornment before a source tab
|
||||
contributes to the visible logical column, the tab still ends at the next
|
||||
8-column stop, and source/adornment hit gravity remains deterministic.
|
||||
12. **Minimap parity:** leading and interior tabs use 8-column stops; width-2
|
||||
and zero-width Unicode affect minimap logical columns by 2 and 0 rather
|
||||
than 1.
|
||||
13. **Edit freshness:** inserting or deleting a tab on a visible line updates
|
||||
shaping, caret position, hit testing, styles/decorations, and minimap shape
|
||||
on the next normal refresh without switching buffers or forcing a full
|
||||
rebuild.
|
||||
14. **No scope creep:** `pmacs.config` gains no tab-width key, and this work
|
||||
changes no `PROTOCOL_VERSION`, wire message shape, or negotiation rule.
|
||||
15. **Quality gates:** focused default/Lua 5.4 tests, the touched acceptance
|
||||
suite, both GPU unit and required hardware-backed tests, the standard
|
||||
project gates, workspace sweep, and `git diff --check` pass.
|
||||
|
||||
## Verification plan
|
||||
|
||||
Focused checks should exercise the shared arithmetic and the two real render
|
||||
paths rather than inspecting source text:
|
||||
|
||||
- core unit tests for valid-prefix handling, Unicode widths, and tab starts at
|
||||
0/7/8;
|
||||
- existing and extended `TextView`, highlight, diagnostic, completion, and
|
||||
overlay tests using byte ranges that cross tabs;
|
||||
- GPU projection-map tests for tab expansion and both mapping directions;
|
||||
- GPU layout/offscreen tests for caret, selection/diagnostic geometry,
|
||||
a soft-wrap boundary inside an expanded tab, adornments, and edit freshness;
|
||||
- minimap shape tests for tabs and Unicode; and
|
||||
- `tests/tab_width_acceptance.rs` rendering one tabbed fixture through the
|
||||
core-facing path while the GPU suite proves the frontend projection.
|
||||
|
||||
Required commands before a PR:
|
||||
|
||||
```text
|
||||
cargo fmt --check
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
cargo test --lib
|
||||
cargo test --lib --features crdt
|
||||
cargo test --no-default-features --features lua54 --lib
|
||||
cargo test --test tab_width_acceptance
|
||||
cargo test --test m4_acceptance -- --skip basedpyright
|
||||
PMACS_REQUIRE_GPU=1 cargo test -p pmacs-gpu
|
||||
cargo test --workspace -- --skip basedpyright
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Run strict Clippy as its own command. Any known timing-only failure must be
|
||||
rerun isolated per `docs/agent-handoff.md`; a rerun is evidence only when the
|
||||
failure matches a documented flaky test.
|
||||
|
||||
## Expected files
|
||||
|
||||
- `pmacs-protocol/src/lib.rs` - canonical tab-stop rendering constant and
|
||||
protocol-level documentation.
|
||||
- `src/display_width.rs` and `src/lib.rs` - shared core display-column logic and
|
||||
module export.
|
||||
- `src/text_view.rs`, `src/highlight.rs`, `src/diag.rs`, `src/completion.rs`,
|
||||
and `src/overlay.rs` - remove duplicated arithmetic and consume the helper.
|
||||
- `pmacs-gpu/Cargo.toml`, `Cargo.lock`, and `pmacs-gpu/src/main.rs` - direct
|
||||
Unicode-width dependency, tab projection/provenance, geometry mapping,
|
||||
minimap parity, and focused tests.
|
||||
- `tests/tab_width_acceptance.rs` - focused observable TUI/core parity across
|
||||
plain text and byte-addressed overlays.
|
||||
- `docs/agent-handoff.md`, `docs/active-work.md`,
|
||||
`docs/side-quest-backlog.md`, and this framing document - updated only after
|
||||
implementation is proven and published according to their protocols.
|
||||
|
||||
No Lua runtime, config-registry, syntax-query, theme, or serialized protocol
|
||||
file should need a behavior change.
|
||||
|
|
@ -61,3 +61,4 @@ pmacs-protocol = { version = "1.0.0", path = "../pmacs-protocol" }
|
|||
pollster = "0.4.0"
|
||||
wgpu = "29.0.3"
|
||||
winit = "0.30.13"
|
||||
unicode-width = "0.2"
|
||||
|
|
|
|||
|
|
@ -42,10 +42,11 @@ use pmacs_protocol::{
|
|||
MAX_STATUSLINE_FACE_BYTES, MAX_STATUSLINE_PROVIDERS, MAX_STATUSLINE_SEGMENT_BYTES,
|
||||
MAX_STATUSLINE_TOTAL_TEXT_BYTES, MenuPromptRow, Modifiers, MouseButton as ProtocolMouseButton,
|
||||
MouseKind as ProtocolMouseKind, PointerKind, SelectionSnapshot, StatuslineSegment,
|
||||
StyleSegment, StyleSpan, TerminalFrame, UnderlineStyle,
|
||||
StyleSegment, StyleSpan, TAB_STOP_COLUMNS, TerminalFrame, UnderlineStyle,
|
||||
cell::{Color as CellColor, Style as CellStyle},
|
||||
is_builtin_pair_char, is_modeline_face_name,
|
||||
};
|
||||
use unicode_width::UnicodeWidthChar;
|
||||
use wgpu::MultisampleState;
|
||||
use winit::application::ApplicationHandler;
|
||||
use winit::event::{ElementState, WindowEvent};
|
||||
|
|
@ -933,9 +934,9 @@ struct State {
|
|||
current_line_char_starts: Vec<u64>,
|
||||
/// Code-shape data used to give the minimap horizontal structure
|
||||
/// even though `FileStyleSummary` carries only one dominant style
|
||||
/// per line. Refreshed when a new summary lands, keeping this
|
||||
/// cache in cadence with the debounced minimap data rather than
|
||||
/// rebuilding it for every typed byte.
|
||||
/// per line. Summary replacement rebuilds the table; accepted text
|
||||
/// edits update the affected line immediately (or rebuild after
|
||||
/// structural/batched edits).
|
||||
current_line_shapes: Vec<MinimapLineShape>,
|
||||
/// Local CRDT replica seeded by `BufferSnapshot`. `None` in
|
||||
/// hello-world mode or before the first snapshot arrives in
|
||||
|
|
@ -3085,6 +3086,7 @@ impl State {
|
|||
if edits.is_empty() {
|
||||
return Ok(edits);
|
||||
}
|
||||
self.refresh_minimap_shapes_after_edits(&edits, line_count_before);
|
||||
self.translate_cached_anchors(&edits);
|
||||
// A newline edit can cross a gutter digit boundary (9 -> 10,
|
||||
// 99 -> 100). Synchronize the painter-derived code width
|
||||
|
|
@ -3115,6 +3117,36 @@ impl State {
|
|||
}
|
||||
}
|
||||
|
||||
/// Keep minimap horizontal geometry in lock-step with accepted text
|
||||
/// edits instead of waiting for the next debounced style summary.
|
||||
/// The common one-line edit updates one cached shape; line-structure
|
||||
/// or batched edits rebuild the table because their intermediate
|
||||
/// coordinates need not describe the final line partition.
|
||||
fn refresh_minimap_shapes_after_edits(
|
||||
&mut self,
|
||||
edits: &[TextProjectionEdit],
|
||||
line_count_before: usize,
|
||||
) {
|
||||
if edits.len() == 1
|
||||
&& self.current_line_starts.len() == line_count_before
|
||||
&& self.current_line_shapes.len() == self.current_line_starts.len()
|
||||
{
|
||||
let line = self
|
||||
.current_line_starts
|
||||
.partition_point(|&start| start <= edits[0].start)
|
||||
.saturating_sub(1);
|
||||
let start = self.current_line_starts[line] as usize;
|
||||
let end = self
|
||||
.current_line_starts
|
||||
.get(line + 1)
|
||||
.map_or(self.current_text.len(), |next| *next as usize - 1);
|
||||
self.current_line_shapes[line] = minimap_line_shape(&self.current_text[start..end]);
|
||||
} else {
|
||||
self.current_line_shapes = minimap_line_shapes(&self.current_text);
|
||||
}
|
||||
self.minimap_cache = None;
|
||||
}
|
||||
|
||||
/// Drop journal entries already reflected in a producer frame
|
||||
/// stamped `generation` — see the `unconfirmed_edits` field docs.
|
||||
fn prune_unconfirmed_edits(&mut self, generation: u64) {
|
||||
|
|
@ -3184,6 +3216,8 @@ impl State {
|
|||
let (line_starts, line_char_starts) = line_offset_tables(text);
|
||||
self.current_line_starts = line_starts;
|
||||
self.current_line_char_starts = line_char_starts;
|
||||
self.current_line_shapes = minimap_line_shapes(text);
|
||||
self.minimap_cache = None;
|
||||
let geometry_changed = self.sync_buffer_dimensions();
|
||||
self.reshape();
|
||||
if geometry_changed && caret_was_painted {
|
||||
|
|
@ -7042,46 +7076,21 @@ impl State {
|
|||
})
|
||||
}
|
||||
|
||||
/// Map an absolute source `byte` to `(slice line index, projected
|
||||
/// byte offset within that shaped line)` by inverting the line's
|
||||
/// `line_chunk_cache` projection (framing Q#F6): source bytes are
|
||||
/// not projected bytes once inline adornments inject text. An
|
||||
/// adornment anchor maps to the EARLIEST projected boundary — the
|
||||
/// current left-gravity caret placement, before the injected
|
||||
/// text. `None` when the byte's source line is outside the shaped
|
||||
/// slice.
|
||||
/// Map an absolute source byte to `(slice line index, projected
|
||||
/// byte offset within that shaped line)` through the same reusable
|
||||
/// chunk mapping used by decoration geometry.
|
||||
/// Adornments retain left gravity, while a source tab's two byte
|
||||
/// boundaries map to the leading and trailing edges of all of its
|
||||
/// projected spaces.
|
||||
fn code_byte_to_projected(&self, byte: u64) -> Option<(usize, usize)> {
|
||||
let line_idx = self
|
||||
.current_line_starts
|
||||
.partition_point(|&s| s <= byte)
|
||||
.saturating_sub(1);
|
||||
let slice_i = line_idx.checked_sub(self.shaped_top)?;
|
||||
if slice_i >= self.line_chunk_cache.len() {
|
||||
return None;
|
||||
}
|
||||
let chunks = self.line_chunk_cache.get(slice_i)?;
|
||||
let rel = byte - self.current_line_starts[line_idx];
|
||||
let mut projected = 0usize;
|
||||
for chunk in &self.line_chunk_cache[slice_i] {
|
||||
match chunk.source {
|
||||
ChunkSource::Source { start } => {
|
||||
let len = chunk.text.len() as u64;
|
||||
if rel >= start && rel < start + len {
|
||||
return Some((slice_i, projected + (rel - start) as usize));
|
||||
}
|
||||
}
|
||||
ChunkSource::Adornment { anchor } => {
|
||||
// Source chunks tile the line, so reaching an
|
||||
// adornment chunk unmatched means the byte sits at
|
||||
// its anchor boundary (or past line end).
|
||||
if rel <= anchor {
|
||||
return Some((slice_i, projected));
|
||||
}
|
||||
}
|
||||
}
|
||||
projected += chunk.text.len();
|
||||
}
|
||||
// Line end (the `\n` position, or EOF).
|
||||
Some((slice_i, projected))
|
||||
source_to_projected(chunks, rel).map(|projected| (slice_i, projected as usize))
|
||||
}
|
||||
|
||||
/// Convert an absolute source byte to a cursor cosmic-text can
|
||||
|
|
@ -7227,11 +7236,11 @@ impl State {
|
|||
}
|
||||
|
||||
/// Push one rect per visual line whose glyphs overlap the
|
||||
/// buffer-absolute byte range `[lo, hi)`, spanning the matching
|
||||
/// glyphs' horizontal extent. A range crossing visual-line
|
||||
/// boundaries (wrapped or multi-line) fans out into one rect per
|
||||
/// run. `line_offsets[run.line_i]` rebases the run's line-relative
|
||||
/// glyph offsets into buffer-absolute space for the comparison.
|
||||
/// slice-relative source byte range `[lo, hi)`, spanning the
|
||||
/// matching projected glyphs' horizontal extent. Each source-line
|
||||
/// intersection is mapped through its cached chunks first, so a
|
||||
/// source tab covers every expanded space even when a soft wrap
|
||||
/// divides those spaces between visual runs.
|
||||
fn push_glyph_extent_rects(
|
||||
&self,
|
||||
rects: &mut Vec<MinimapRect>,
|
||||
|
|
@ -7248,12 +7257,30 @@ impl State {
|
|||
let text_left = self.text_left();
|
||||
for run in self.buffer.layout_runs() {
|
||||
let line_base = line_offsets.get(run.line_i).copied().unwrap_or(0);
|
||||
let line_end = line_offsets
|
||||
.get(run.line_i + 1)
|
||||
.copied()
|
||||
.unwrap_or(self.view_range.1 - self.view_range.0);
|
||||
let source_lo = lo.max(line_base);
|
||||
let source_hi = hi.min(line_end);
|
||||
if source_hi <= source_lo {
|
||||
continue;
|
||||
}
|
||||
let Some(chunks) = self.line_chunk_cache.get(run.line_i) else {
|
||||
continue;
|
||||
};
|
||||
let Some(projected_lo) = source_to_projected(chunks, source_lo - line_base) else {
|
||||
continue;
|
||||
};
|
||||
let Some(projected_hi) = source_to_projected(chunks, source_hi - line_base) else {
|
||||
continue;
|
||||
};
|
||||
let mut min_x: Option<f32> = None;
|
||||
let mut max_x: Option<f32> = None;
|
||||
for glyph in run.glyphs {
|
||||
let g_start = line_base + glyph.start as u64;
|
||||
let g_end = line_base + glyph.end as u64;
|
||||
if g_end <= lo || g_start >= hi {
|
||||
let g_start = glyph.start as u64;
|
||||
let g_end = glyph.end as u64;
|
||||
if g_end <= projected_lo || g_start >= projected_hi {
|
||||
continue;
|
||||
}
|
||||
let x0 = glyph.x;
|
||||
|
|
@ -7315,6 +7342,8 @@ struct RichChunk {
|
|||
enum ChunkSource {
|
||||
/// Verbatim source text starting at this slice byte offset.
|
||||
Source { start: u64 },
|
||||
/// One source tab byte expanded into one or more projected spaces.
|
||||
SourceTab { start: u64 },
|
||||
/// Injected adornment text (inlay hint) anchored at this slice
|
||||
/// byte offset. Hits inside it snap to the anchor.
|
||||
Adornment { anchor: u64 },
|
||||
|
|
@ -7355,9 +7384,10 @@ fn build_hit_runs(chunks: &[RichChunk]) -> (Vec<ProjectedRun>, Vec<u64>) {
|
|||
(runs, line_starts)
|
||||
}
|
||||
|
||||
/// Map a projected byte offset back to a slice-relative source byte
|
||||
/// (Q#M2). Hits inside an adornment run snap to its anchor; offsets
|
||||
/// past the last run clamp to its end.
|
||||
/// Map a projected byte offset back to a slice-relative source byte.
|
||||
/// A source tab's leading boundary maps before the byte; every
|
||||
/// boundary inside its expanded spaces (including the trailing edge)
|
||||
/// maps after it. Adornments snap to their left-gravity anchor.
|
||||
fn projected_to_source(runs: &[ProjectedRun], projected: u64) -> Option<u64> {
|
||||
if runs.is_empty() {
|
||||
return None;
|
||||
|
|
@ -7369,10 +7399,48 @@ fn projected_to_source(runs: &[ProjectedRun], projected: u64) -> Option<u64> {
|
|||
let within = projected.saturating_sub(run.projected_start).min(run.len);
|
||||
match run.source {
|
||||
ChunkSource::Source { start } => Some(start + within),
|
||||
ChunkSource::SourceTab { start } => Some(start + u64::from(within > 0)),
|
||||
ChunkSource::Adornment { anchor } => Some(anchor),
|
||||
}
|
||||
}
|
||||
|
||||
/// Map a slice-relative source boundary into projected byte space.
|
||||
/// This is the inverse boundary policy shared by caret placement and
|
||||
/// horizontal decoration geometry. At an adornment anchor the earliest
|
||||
/// projected boundary wins, preserving left gravity.
|
||||
fn source_to_projected(chunks: &[RichChunk], source: u64) -> Option<u64> {
|
||||
let mut projected = 0u64;
|
||||
for chunk in chunks {
|
||||
let len = chunk.text.len() as u64;
|
||||
match chunk.source {
|
||||
ChunkSource::Source { start } => {
|
||||
if source <= start {
|
||||
return Some(projected);
|
||||
}
|
||||
let end = start + len;
|
||||
if source <= end {
|
||||
return Some(projected + source - start);
|
||||
}
|
||||
}
|
||||
ChunkSource::SourceTab { start } => {
|
||||
if source <= start {
|
||||
return Some(projected);
|
||||
}
|
||||
if source <= start + 1 {
|
||||
return Some(projected + len);
|
||||
}
|
||||
}
|
||||
ChunkSource::Adornment { anchor } => {
|
||||
if source <= anchor {
|
||||
return Some(projected);
|
||||
}
|
||||
}
|
||||
}
|
||||
projected += len;
|
||||
}
|
||||
(!chunks.is_empty()).then_some(projected)
|
||||
}
|
||||
|
||||
fn minimap_left(surface_width: u32) -> Option<f32> {
|
||||
if surface_width < MINIMAP_MIN_SURFACE_WIDTH {
|
||||
return None;
|
||||
|
|
@ -7699,9 +7767,10 @@ fn minimap_line_shape(line: &str) -> MinimapLineShape {
|
|||
|
||||
fn advance_minimap_col(col: usize, ch: char) -> usize {
|
||||
if ch == '\t' {
|
||||
((col / 4) + 1) * 4
|
||||
let tab_stop = TAB_STOP_COLUMNS as usize;
|
||||
col + tab_stop - col % tab_stop
|
||||
} else {
|
||||
col + 1
|
||||
col + UnicodeWidthChar::width(ch).unwrap_or(0)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -8639,7 +8708,89 @@ fn projected_rich_chunks(
|
|||
source: ChunkSource::Source { start: 0 },
|
||||
});
|
||||
}
|
||||
chunks
|
||||
expand_chunk_tabs(chunks)
|
||||
}
|
||||
|
||||
/// Expand display tabs after source styling and adornment insertion.
|
||||
/// Chunks without tabs are moved through unchanged. A chunk containing
|
||||
/// tabs is split only at those bytes; every emitted space keeps the
|
||||
/// original color, while source tabs gain explicit provenance.
|
||||
fn expand_chunk_tabs(chunks: Vec<RichChunk>) -> Vec<RichChunk> {
|
||||
let mut expanded = Vec::with_capacity(chunks.len());
|
||||
let mut column = 0usize;
|
||||
for chunk in chunks {
|
||||
if !chunk.text.contains('\t') {
|
||||
advance_display_column(&mut column, &chunk.text);
|
||||
expanded.push(chunk);
|
||||
continue;
|
||||
}
|
||||
|
||||
let RichChunk {
|
||||
text,
|
||||
color,
|
||||
source,
|
||||
} = chunk;
|
||||
let mut segment_start = 0usize;
|
||||
for (byte, ch) in text.char_indices() {
|
||||
if ch != '\t' {
|
||||
continue;
|
||||
}
|
||||
if segment_start < byte {
|
||||
let segment = &text[segment_start..byte];
|
||||
advance_display_column(&mut column, segment);
|
||||
expanded.push(RichChunk {
|
||||
text: segment.to_owned(),
|
||||
color,
|
||||
source: offset_chunk_source(source, segment_start as u64),
|
||||
});
|
||||
}
|
||||
let tab_stop = TAB_STOP_COLUMNS as usize;
|
||||
let tab_width = tab_stop - column % tab_stop;
|
||||
expanded.push(RichChunk {
|
||||
text: " ".repeat(tab_width),
|
||||
color,
|
||||
source: match source {
|
||||
ChunkSource::Source { start } => ChunkSource::SourceTab {
|
||||
start: start + byte as u64,
|
||||
},
|
||||
ChunkSource::Adornment { anchor } => ChunkSource::Adornment { anchor },
|
||||
ChunkSource::SourceTab { start } => ChunkSource::SourceTab { start },
|
||||
},
|
||||
});
|
||||
column += tab_width;
|
||||
segment_start = byte + 1;
|
||||
}
|
||||
if segment_start < text.len() {
|
||||
let segment = &text[segment_start..];
|
||||
advance_display_column(&mut column, segment);
|
||||
expanded.push(RichChunk {
|
||||
text: segment.to_owned(),
|
||||
color,
|
||||
source: offset_chunk_source(source, segment_start as u64),
|
||||
});
|
||||
}
|
||||
}
|
||||
expanded
|
||||
}
|
||||
|
||||
fn offset_chunk_source(source: ChunkSource, byte_offset: u64) -> ChunkSource {
|
||||
match source {
|
||||
ChunkSource::Source { start } => ChunkSource::Source {
|
||||
start: start + byte_offset,
|
||||
},
|
||||
ChunkSource::SourceTab { start } => ChunkSource::SourceTab { start },
|
||||
ChunkSource::Adornment { anchor } => ChunkSource::Adornment { anchor },
|
||||
}
|
||||
}
|
||||
|
||||
fn advance_display_column(column: &mut usize, text: &str) {
|
||||
for ch in text.chars() {
|
||||
if ch == '\n' {
|
||||
*column = 0;
|
||||
} else {
|
||||
*column += UnicodeWidthChar::width(ch).unwrap_or(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn renderable_adornment_anchor(adornment: &InlineAdornment, text_len: u64) -> Option<u64> {
|
||||
|
|
@ -9547,6 +9698,109 @@ mod tests {
|
|||
assert_eq!(projected_to_source(&[], 0), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tab_projection_uses_shared_stops_and_unicode_columns() {
|
||||
let projected = |text: &str| {
|
||||
projected_rich_chunks(text, &[], &[])
|
||||
.into_iter()
|
||||
.map(|chunk| chunk.text)
|
||||
.collect::<String>()
|
||||
};
|
||||
|
||||
assert_eq!(projected("\t"), " ", "column 0 advances to 8");
|
||||
assert_eq!(projected("1234567\t"), "1234567 ", "column 7 advances to 8");
|
||||
assert_eq!(
|
||||
projected("12345678\t"),
|
||||
"12345678 ",
|
||||
"column 8 advances to 16"
|
||||
);
|
||||
assert_eq!(
|
||||
projected("界\t\n\u{301}\t"),
|
||||
"界 \n\u{301} ",
|
||||
"wide scalars count as two, zero-width scalars as zero, and newline resets"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tab_projection_preserves_source_and_adornment_provenance_and_style() {
|
||||
let red = CellColor::Rgb(255, 0, 0);
|
||||
let chunks = projected_rich_chunks(
|
||||
"1234567\tX",
|
||||
&[span(7, 8, red)],
|
||||
&[adornment(0, AdornmentPlacement::AtOffset, "\t")],
|
||||
);
|
||||
assert_eq!(
|
||||
chunks
|
||||
.iter()
|
||||
.map(|chunk| chunk.text.as_str())
|
||||
.collect::<String>(),
|
||||
" 1234567 X",
|
||||
"the adornment tab participates in the same logical column stream"
|
||||
);
|
||||
let source_tab = chunks
|
||||
.iter()
|
||||
.find(|chunk| matches!(chunk.source, ChunkSource::SourceTab { start: 7 }))
|
||||
.expect("source tab has a first-class projected run");
|
||||
assert_eq!(source_tab.text, " ");
|
||||
assert_eq!(source_tab.color, cell_color_to_glyphon(red));
|
||||
assert!(
|
||||
chunks.iter().any(
|
||||
|chunk| matches!(chunk.source, ChunkSource::Adornment { anchor: 0 })
|
||||
&& chunk.text == " "
|
||||
),
|
||||
"adornment tabs expand without pretending to be source bytes"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tab_projection_moves_chunks_without_tabs_unchanged() {
|
||||
let text = String::from("wide 界 and plain");
|
||||
let allocation = text.as_ptr();
|
||||
let chunks = expand_chunk_tabs(vec![RichChunk {
|
||||
text,
|
||||
color: None,
|
||||
source: ChunkSource::Source { start: 0 },
|
||||
}]);
|
||||
|
||||
assert_eq!(chunks.len(), 1);
|
||||
assert_eq!(chunks[0].text.as_ptr(), allocation);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn source_tab_projection_boundaries_are_bidirectional() {
|
||||
let chunks = projected_rich_chunks("\tX", &[], &[]);
|
||||
let (runs, _) = build_hit_runs(&chunks);
|
||||
|
||||
assert_eq!(source_to_projected(&chunks, 0), Some(0));
|
||||
assert_eq!(source_to_projected(&chunks, 1), Some(8));
|
||||
assert_eq!(source_to_projected(&chunks, 2), Some(9));
|
||||
assert_eq!(projected_to_source(&runs, 0), Some(0));
|
||||
for projected in 1..=8 {
|
||||
assert_eq!(
|
||||
projected_to_source(&runs, projected),
|
||||
Some(1),
|
||||
"projected boundary {projected} inside the tab maps after its source byte"
|
||||
);
|
||||
}
|
||||
assert_eq!(projected_to_source(&runs, 9), Some(2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adornment_tab_keeps_left_gravity_in_source_mapping() {
|
||||
let chunks = projected_rich_chunks(
|
||||
"X",
|
||||
&[],
|
||||
&[adornment(0, AdornmentPlacement::AtOffset, "\t")],
|
||||
);
|
||||
let (runs, _) = build_hit_runs(&chunks);
|
||||
|
||||
assert_eq!(source_to_projected(&chunks, 0), Some(0));
|
||||
assert_eq!(source_to_projected(&chunks, 1), Some(9));
|
||||
for projected in 0..8 {
|
||||
assert_eq!(projected_to_source(&runs, projected), Some(0));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn optimistic_delete_range_covers_single_codepoints_only() {
|
||||
let none = Modifiers::NONE;
|
||||
|
|
@ -10108,6 +10362,27 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn minimap_columns_match_code_tab_and_unicode_widths() {
|
||||
assert_eq!(
|
||||
minimap_line_shapes("\tX\n1234567\tX\n界\u{301}\tX"),
|
||||
vec![
|
||||
MinimapLineShape {
|
||||
indent_cols: 8,
|
||||
content_cols: 1,
|
||||
},
|
||||
MinimapLineShape {
|
||||
indent_cols: 0,
|
||||
content_cols: 9,
|
||||
},
|
||||
MinimapLineShape {
|
||||
indent_cols: 0,
|
||||
content_cols: 9,
|
||||
},
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn minimap_rects_encode_six_vertices_per_quad() {
|
||||
let rect = MinimapRect {
|
||||
|
|
@ -12564,6 +12839,141 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn source_tab_caret_uses_projected_leading_and_trailing_boundaries() {
|
||||
let Some(mut state) = headless_or_skip(320, 240, "\tX") else {
|
||||
return;
|
||||
};
|
||||
let bid = BufferId::next();
|
||||
state.current_buffer_id = Some(bid);
|
||||
state.reshape();
|
||||
|
||||
state.own_cursor = Some(OwnCursor {
|
||||
buffer_id: bid,
|
||||
byte: 0,
|
||||
});
|
||||
let before = state.caret_rect().expect("caret before tab").x;
|
||||
state.own_cursor = Some(OwnCursor {
|
||||
buffer_id: bid,
|
||||
byte: 1,
|
||||
});
|
||||
let after = state.caret_rect().expect("caret after tab").x;
|
||||
assert!(
|
||||
after - before > 7.0 * state.mono_advance(),
|
||||
"one source byte must span all eight projected spaces"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn source_tab_hit_testing_uses_projected_space_boundaries() {
|
||||
let Some(mut state) = headless_or_skip(320, 240, "\tX") else {
|
||||
return;
|
||||
};
|
||||
state.current_buffer_id = Some(BufferId::next());
|
||||
state.reshape();
|
||||
let advance = state.mono_advance();
|
||||
let y = f64::from(TEXT_TOP + state.fm.code_line_height() / 2.0);
|
||||
|
||||
assert_eq!(
|
||||
state.hit_test_source_byte(f64::from(state.text_left()), y),
|
||||
Some(0),
|
||||
"the projected leading edge maps before the source tab"
|
||||
);
|
||||
assert_eq!(
|
||||
state.hit_test_source_byte(f64::from(state.text_left() + advance * 2.5), y,),
|
||||
Some(1),
|
||||
"a hit inside the expanded spaces maps after the source tab"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tab_decoration_geometry_covers_spaces_split_by_soft_wrap() {
|
||||
let Some(mut state) = headless_or_skip(64, 240, "\tX") else {
|
||||
return;
|
||||
};
|
||||
let bid = BufferId::next();
|
||||
state.current_buffer_id = Some(bid);
|
||||
state.current_decorations = vec![Decoration {
|
||||
range: ByteRange { start: 0, end: 1 },
|
||||
kind: DecorationKind::Selection,
|
||||
}];
|
||||
state.reshape();
|
||||
assert!(
|
||||
state
|
||||
.buffer
|
||||
.layout_runs()
|
||||
.filter(|run| run.line_i == 0)
|
||||
.count()
|
||||
> 1,
|
||||
"precondition: the eight projected spaces wrap"
|
||||
);
|
||||
|
||||
let line_offsets = line_byte_offsets(&state.current_text);
|
||||
let mut rects = Vec::new();
|
||||
state.collect_own_decoration_rects(
|
||||
&mut rects,
|
||||
&line_offsets,
|
||||
state.view_range.0,
|
||||
state.view_range.1,
|
||||
);
|
||||
assert!(
|
||||
rects.len() > 1,
|
||||
"the source tab selection must fan out across wrapped visual runs"
|
||||
);
|
||||
assert!(
|
||||
rects.iter().all(|rect| rect.w > 0.0),
|
||||
"every wrapped piece must retain horizontal geometry"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn visible_line_tab_edit_refreshes_cached_projection() {
|
||||
let Some(mut state) = headless_or_skip(320, 240, "aX") else {
|
||||
return;
|
||||
};
|
||||
state.minimap_cache = Some(((0, 0, 0, 0), vec![1]));
|
||||
let edits = state
|
||||
.apply_loro_text_delta_batches(&[vec![
|
||||
loro::TextDelta::Retain {
|
||||
retain: 1,
|
||||
attributes: None,
|
||||
},
|
||||
loro::TextDelta::Insert {
|
||||
insert: "\t".to_owned(),
|
||||
attributes: None,
|
||||
},
|
||||
]])
|
||||
.expect("visible edit applies");
|
||||
|
||||
assert_eq!(
|
||||
edits,
|
||||
vec![TextProjectionEdit {
|
||||
start: 1,
|
||||
old_end: 1,
|
||||
inserted_len: 1,
|
||||
}]
|
||||
);
|
||||
assert_eq!(state.buffer.lines[0].text(), "a X");
|
||||
assert_eq!(
|
||||
state.current_line_shapes[0],
|
||||
MinimapLineShape {
|
||||
indent_cols: 0,
|
||||
content_cols: 9,
|
||||
},
|
||||
"the minimap shape must refresh in the same edit transaction"
|
||||
);
|
||||
assert!(
|
||||
state.minimap_cache.is_none(),
|
||||
"text geometry changes must invalidate cached minimap vertices"
|
||||
);
|
||||
assert!(
|
||||
state.line_chunk_cache[0]
|
||||
.iter()
|
||||
.any(|chunk| matches!(chunk.source, ChunkSource::SourceTab { start: 1 })),
|
||||
"the incremental code-line cache must immediately carry tab provenance"
|
||||
);
|
||||
}
|
||||
|
||||
/// Acceptance 11 — the `CursorByte` arm follows into a wrapped
|
||||
/// continuation run (the pre-existing source-line-only hole): the
|
||||
/// follow lands as a sub-line residual, normalized to slice-local
|
||||
|
|
|
|||
|
|
@ -17,6 +17,8 @@
|
|||
//! - The full message envelopes: `InstanceMessage`, `FrontendEvent`,
|
||||
//! `GoodbyeReason`, capability structs, `PresenceUpdate`, etc.
|
||||
//! - The optional `CrdtOp` wire variant (feature-gated on `crdt`).
|
||||
//! - [`TAB_STOP_COLUMNS`], the shared logical width used when frontends
|
||||
//! project raw buffer tabs for display.
|
||||
//!
|
||||
//! What does NOT live here:
|
||||
//! - `crate::cell::CellGrid` and `crate::cell::diff()` (rendering
|
||||
|
|
@ -41,6 +43,12 @@ pub mod message;
|
|||
pub mod terminal;
|
||||
pub mod transport;
|
||||
|
||||
/// Logical display columns between fixed buffer-text tab stops.
|
||||
///
|
||||
/// Semantic frames keep tabs as source bytes; every frontend expands them
|
||||
/// only in its display projection so protocol byte ranges remain unchanged.
|
||||
pub const TAB_STOP_COLUMNS: u32 = 8;
|
||||
|
||||
pub use cell::{
|
||||
Attachment, Cell, CellCoord, CellSize, Color, DiffSpan, Glyph, Style, UnderlineStyle,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ use unicode_width::UnicodeWidthChar;
|
|||
|
||||
use crate::buffer::{Buffer, BufferId};
|
||||
use crate::cell::{CellCoord, CellGrid, Color, Glyph, Style};
|
||||
use crate::display_width::byte_to_column;
|
||||
use crate::rope::Position;
|
||||
use crate::view::{View, Viewport};
|
||||
|
||||
|
|
@ -589,10 +590,6 @@ pub(crate) const POPUP_MAX_ROWS: u32 = 10;
|
|||
/// Minimum popup width in cells (glyph column + a readable label).
|
||||
const POPUP_MIN_WIDTH: u32 = 12;
|
||||
|
||||
/// Tab-stop width in display columns, matching [`crate::diag`] /
|
||||
/// [`crate::text_view`].
|
||||
const TAB_WIDTH: u32 = 8;
|
||||
|
||||
/// Style for the currently-selected row (reverse video so it pops on
|
||||
/// any base palette).
|
||||
fn selected_style() -> Style {
|
||||
|
|
@ -663,21 +660,9 @@ impl CompletionView {
|
|||
}
|
||||
}
|
||||
|
||||
/// Display column of `byte_end` within `line_bytes` (tab-aware,
|
||||
/// UTF-8-aware). The completion twin of the diagnostic underline's
|
||||
/// column resolution.
|
||||
/// Display column of `byte_end` within `line_bytes`.
|
||||
fn display_col_for_byte(line_bytes: &[u8], byte_end: u32) -> u32 {
|
||||
let end = (byte_end as usize).min(line_bytes.len());
|
||||
let text = String::from_utf8_lossy(&line_bytes[..end]);
|
||||
let mut col = 0u32;
|
||||
for ch in text.chars() {
|
||||
if ch == '\t' {
|
||||
col += TAB_WIDTH - (col % TAB_WIDTH);
|
||||
} else {
|
||||
col += char_display_width(ch);
|
||||
}
|
||||
}
|
||||
col
|
||||
byte_to_column(line_bytes, byte_end as usize)
|
||||
}
|
||||
|
||||
/// Resolved popup rectangle, in window-relative cells.
|
||||
|
|
@ -811,7 +796,7 @@ fn paint_popup_row(
|
|||
if col >= width {
|
||||
break;
|
||||
}
|
||||
let cw = char_display_width(ch);
|
||||
let cw = UnicodeWidthChar::width(ch).unwrap_or(0) as u32;
|
||||
if cw == 0 {
|
||||
continue;
|
||||
}
|
||||
|
|
@ -879,10 +864,6 @@ impl View for CompletionView {
|
|||
}
|
||||
}
|
||||
|
||||
fn char_display_width(ch: char) -> u32 {
|
||||
UnicodeWidthChar::width(ch).unwrap_or(0) as u32
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
|
|||
43
src/diag.rs
43
src/diag.rs
|
|
@ -32,10 +32,10 @@ use std::collections::HashMap;
|
|||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use serde_json::Value;
|
||||
use unicode_width::UnicodeWidthChar;
|
||||
|
||||
use crate::buffer::Buffer;
|
||||
use crate::cell::{CellCoord, CellGrid, Color, Glyph, Style, UnderlineStyle};
|
||||
use crate::display_width::byte_range_to_columns;
|
||||
use crate::overlay::merge_styles;
|
||||
use crate::view::{View, Viewport};
|
||||
|
||||
|
|
@ -388,10 +388,6 @@ pub fn make_shared_store() -> SharedDiagStore {
|
|||
// View
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Tab-stop width in display columns, matching
|
||||
/// [`crate::text_view`] and [`crate::highlight`].
|
||||
const TAB_WIDTH: u32 = 8;
|
||||
|
||||
/// The RESOLVED severity color (themes arc Q#TH5): the `ui.diag.*`
|
||||
/// face's `fg` when a face is set with a concrete color, else the
|
||||
/// built-in [`DiagnosticSeverity::underline_color`]. The diag family
|
||||
|
|
@ -665,8 +661,7 @@ fn paint_line_markers(
|
|||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Shared helpers (mirror highlight.rs; kept private here to avoid
|
||||
// cross-module coupling on internal helpers)
|
||||
// Line lookup helpers shared with the completion overlay.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
pub(crate) fn compute_line_offsets(source: &[u8]) -> Vec<u32> {
|
||||
|
|
@ -699,40 +694,10 @@ pub(crate) fn line_at_offset(line_offsets: &[u32], offset: u32) -> u32 {
|
|||
fn underline_cols_for_line(line_bytes: &[u8], byte_start: u32, byte_end: u32) -> (u32, u32) {
|
||||
if byte_end <= byte_start {
|
||||
let (anchor, _) =
|
||||
byte_range_to_display_cols(line_bytes, byte_start as usize, byte_start as usize);
|
||||
byte_range_to_columns(line_bytes, byte_start as usize, byte_start as usize);
|
||||
(anchor, anchor + 1)
|
||||
} else {
|
||||
byte_range_to_display_cols(line_bytes, byte_start as usize, byte_end as usize)
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn byte_range_to_display_cols(
|
||||
line_bytes: &[u8],
|
||||
byte_start: usize,
|
||||
byte_end: usize,
|
||||
) -> (u32, u32) {
|
||||
let bs = byte_start.min(line_bytes.len());
|
||||
let be = byte_end.min(line_bytes.len());
|
||||
let display_to = |upto: usize| -> u32 {
|
||||
let mut take = upto.min(line_bytes.len());
|
||||
while take > 0 && std::str::from_utf8(&line_bytes[..take]).is_err() {
|
||||
take -= 1;
|
||||
}
|
||||
let s = std::str::from_utf8(&line_bytes[..take]).unwrap_or("");
|
||||
let mut col: u32 = 0;
|
||||
for ch in s.chars() {
|
||||
col += char_display_width(ch, col);
|
||||
}
|
||||
col
|
||||
};
|
||||
(display_to(bs), display_to(be))
|
||||
}
|
||||
|
||||
fn char_display_width(ch: char, current_col: u32) -> u32 {
|
||||
if ch == '\t' {
|
||||
TAB_WIDTH - (current_col % TAB_WIDTH)
|
||||
} else {
|
||||
UnicodeWidthChar::width(ch).unwrap_or(0) as u32
|
||||
byte_range_to_columns(line_bytes, byte_start as usize, byte_end as usize)
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,106 @@
|
|||
// display_width.rs --- Shared byte-to-display-column accounting.
|
||||
|
||||
//! Allocation-free display-column helpers shared by text renderers.
|
||||
//!
|
||||
//! Source positions remain byte-addressed. Tabs are expanded only while
|
||||
//! projecting those bytes into display columns, using the protocol-wide tab
|
||||
//! stop. Offsets are clamped to the supplied slice and offsets inside a UTF-8
|
||||
//! code point resolve to the preceding complete-code-point boundary.
|
||||
|
||||
use unicode_width::UnicodeWidthChar;
|
||||
|
||||
/// Advance `column` past one character.
|
||||
///
|
||||
/// A tab reaches the next protocol tab stop; all other characters use their
|
||||
/// Unicode terminal width. Control and zero-width characters do not advance.
|
||||
#[must_use]
|
||||
pub fn advance_char(column: u32, ch: char) -> u32 {
|
||||
let width = if ch == '\t' {
|
||||
pmacs_protocol::TAB_STOP_COLUMNS - (column % pmacs_protocol::TAB_STOP_COLUMNS)
|
||||
} else {
|
||||
UnicodeWidthChar::width(ch).unwrap_or(0) as u32
|
||||
};
|
||||
column.saturating_add(width)
|
||||
}
|
||||
|
||||
/// Display width of the valid UTF-8 prefix of `bytes`.
|
||||
///
|
||||
/// Invalid input is conservatively truncated at the first invalid byte. This
|
||||
/// also floors a trailing partial code point without allocating or replacing
|
||||
/// source bytes.
|
||||
#[must_use]
|
||||
pub fn valid_prefix_width(bytes: &[u8]) -> u32 {
|
||||
let valid_len = match std::str::from_utf8(bytes) {
|
||||
Ok(_) => bytes.len(),
|
||||
Err(error) => error.valid_up_to(),
|
||||
};
|
||||
let text = std::str::from_utf8(&bytes[..valid_len]).expect("valid_up_to is a UTF-8 boundary");
|
||||
text.chars().fold(0, advance_char)
|
||||
}
|
||||
|
||||
/// Display column at the clamped byte boundary `offset`.
|
||||
///
|
||||
/// If `offset` splits a code point, the result is the column at that code
|
||||
/// point's leading boundary.
|
||||
#[must_use]
|
||||
pub fn byte_to_column(bytes: &[u8], offset: usize) -> u32 {
|
||||
valid_prefix_width(&bytes[..offset.min(bytes.len())])
|
||||
}
|
||||
|
||||
/// Display-column endpoints for the half-open byte range `[start, end)`.
|
||||
///
|
||||
/// Each endpoint is independently clamped and conservatively floored to a
|
||||
/// complete UTF-8 boundary.
|
||||
#[must_use]
|
||||
pub fn byte_range_to_columns(bytes: &[u8], start: usize, end: usize) -> (u32, u32) {
|
||||
(byte_to_column(bytes, start), byte_to_column(bytes, end))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn tabs_advance_at_zero_before_stop_and_on_stop() {
|
||||
assert_eq!(advance_char(0, '\t'), 8);
|
||||
assert_eq!(advance_char(7, '\t'), 8);
|
||||
assert_eq!(advance_char(8, '\t'), 16);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unicode_widths_include_wide_and_zero_width_characters() {
|
||||
assert_eq!(advance_char(3, '中'), 5);
|
||||
assert_eq!(advance_char(3, '\u{301}'), 3);
|
||||
assert_eq!(valid_prefix_width("a中\u{301}b".as_bytes()), 4);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn byte_columns_clamp_and_floor_partial_or_invalid_utf8() {
|
||||
let text = "a中b".as_bytes();
|
||||
assert_eq!(byte_to_column(text, 0), 0);
|
||||
assert_eq!(byte_to_column(text, 1), 1);
|
||||
assert_eq!(byte_to_column(text, 2), 1);
|
||||
assert_eq!(byte_to_column(text, 3), 1);
|
||||
assert_eq!(byte_to_column(text, 4), 3);
|
||||
assert_eq!(byte_to_column(text, usize::MAX), 4);
|
||||
|
||||
assert_eq!(valid_prefix_width(b"ab\xffcd"), 2);
|
||||
assert_eq!(byte_to_column(b"ab\xe2\x82", 4), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn byte_ranges_map_half_open_endpoints_with_tab_expansion() {
|
||||
let text = b"a\tb";
|
||||
assert_eq!(byte_range_to_columns(text, 0, 1), (0, 1));
|
||||
assert_eq!(byte_range_to_columns(text, 1, 2), (1, 8));
|
||||
assert_eq!(byte_range_to_columns(text, 2, 3), (8, 9));
|
||||
assert_eq!(byte_range_to_columns(text, 99, 99), (9, 9));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn range_boundaries_inside_codepoints_are_floored() {
|
||||
let text = "a中b".as_bytes();
|
||||
assert_eq!(byte_range_to_columns(text, 2, 3), (1, 1));
|
||||
assert_eq!(byte_range_to_columns(text, 2, 4), (1, 3));
|
||||
}
|
||||
}
|
||||
|
|
@ -33,10 +33,9 @@
|
|||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use unicode_width::UnicodeWidthChar;
|
||||
|
||||
use crate::buffer::Buffer;
|
||||
use crate::cell::{CellCoord, CellGrid, Color, Style, UnderlineStyle};
|
||||
use crate::display_width::byte_range_to_columns;
|
||||
use crate::lsp::SharedLspManager;
|
||||
use crate::overlay::merge_styles;
|
||||
use crate::syntax::{HighlightSpan, ParseTreeBundle, ParseViewHandle, compute_highlight_spans_for};
|
||||
|
|
@ -316,10 +315,6 @@ impl HighlightCache {
|
|||
}
|
||||
}
|
||||
|
||||
/// Tab-stop width in display columns (must match
|
||||
/// [`crate::text_view`]; both views write into the same cell grid).
|
||||
const TAB_WIDTH: u32 = 8;
|
||||
|
||||
/// View that renders syntax highlighting from a tree-sitter parse
|
||||
/// tree, including its injection layers. Composes over
|
||||
/// [`crate::text_view::TextView`] per the M2.9 view-composition
|
||||
|
|
@ -374,7 +369,13 @@ impl SyntaxHighlightView {
|
|||
let Some(query) = layer.highlight_query.as_ref() else {
|
||||
continue;
|
||||
};
|
||||
let spans = compute_highlight_spans_for(query, &layer.tree, source, None);
|
||||
let spans = compute_highlight_spans_for(
|
||||
query,
|
||||
&layer.tree,
|
||||
source,
|
||||
layer.local_facts.as_deref(),
|
||||
None,
|
||||
);
|
||||
if spans.is_empty() {
|
||||
continue;
|
||||
}
|
||||
|
|
@ -476,7 +477,7 @@ impl View for SyntaxHighlightView {
|
|||
let byte_col_start = (s_start - line_start) as usize;
|
||||
let byte_col_end = (s_end - line_start) as usize;
|
||||
let (start_col, end_col) =
|
||||
byte_range_to_display_cols(line_bytes, byte_col_start, byte_col_end);
|
||||
byte_range_to_columns(line_bytes, byte_col_start, byte_col_end);
|
||||
if end_col <= start_col {
|
||||
continue;
|
||||
}
|
||||
|
|
@ -520,40 +521,6 @@ fn line_at_offset(line_offsets: &[u32], offset: u32) -> u32 {
|
|||
}
|
||||
}
|
||||
|
||||
/// Convert a half-open byte-column range `[byte_start, byte_end)`
|
||||
/// inside `line_bytes` to a display-column range. UTF-8 aware; tabs
|
||||
/// expand to the next [`TAB_WIDTH`]-aligned column. Bytes that don't
|
||||
/// form complete codepoints (because the byte range falls inside a
|
||||
/// multi-byte char) are skipped, matching
|
||||
/// [`crate::text_view::TextView::pos_to_display`]'s conservative
|
||||
/// rounding.
|
||||
fn byte_range_to_display_cols(line_bytes: &[u8], byte_start: usize, byte_end: usize) -> (u32, u32) {
|
||||
let bs = byte_start.min(line_bytes.len());
|
||||
let be = byte_end.min(line_bytes.len());
|
||||
let display_to = |upto: usize| -> u32 {
|
||||
// Drop trailing bytes that don't form complete codepoints.
|
||||
let mut take = upto.min(line_bytes.len());
|
||||
while take > 0 && std::str::from_utf8(&line_bytes[..take]).is_err() {
|
||||
take -= 1;
|
||||
}
|
||||
let s = std::str::from_utf8(&line_bytes[..take]).unwrap_or("");
|
||||
let mut col: u32 = 0;
|
||||
for ch in s.chars() {
|
||||
col += char_display_width(ch, col);
|
||||
}
|
||||
col
|
||||
};
|
||||
(display_to(bs), display_to(be))
|
||||
}
|
||||
|
||||
fn char_display_width(ch: char, current_col: u32) -> u32 {
|
||||
if ch == '\t' {
|
||||
TAB_WIDTH - (current_col % TAB_WIDTH)
|
||||
} else {
|
||||
UnicodeWidthChar::width(ch).unwrap_or(0) as u32
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Style equality helper
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
@ -731,7 +698,7 @@ impl View for LspStyleView {
|
|||
if end_b <= start_b {
|
||||
continue;
|
||||
}
|
||||
let (start_col, end_col) = byte_range_to_display_cols(line_bytes, start_b, end_b);
|
||||
let (start_col, end_col) = byte_range_to_columns(line_bytes, start_b, end_b);
|
||||
if end_col <= start_col {
|
||||
continue;
|
||||
}
|
||||
|
|
@ -938,9 +905,9 @@ mod tests {
|
|||
fn byte_range_display_cols_ascii_round_trips() {
|
||||
let line = b"hello world";
|
||||
// "hello" → cols 0..5
|
||||
assert_eq!(byte_range_to_display_cols(line, 0, 5), (0, 5));
|
||||
assert_eq!(byte_range_to_columns(line, 0, 5), (0, 5));
|
||||
// "world" → cols 6..11
|
||||
assert_eq!(byte_range_to_display_cols(line, 6, 11), (6, 11));
|
||||
assert_eq!(byte_range_to_columns(line, 6, 11), (6, 11));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -948,15 +915,15 @@ mod tests {
|
|||
let line = b"\tx";
|
||||
// The full line: tab (0..8) + 'x' (8..9). Byte cols 0..2
|
||||
// map to display cols 0..9.
|
||||
assert_eq!(byte_range_to_display_cols(line, 0, 2), (0, 9));
|
||||
assert_eq!(byte_range_to_columns(line, 0, 2), (0, 9));
|
||||
// Just the tab.
|
||||
assert_eq!(byte_range_to_display_cols(line, 0, 1), (0, 8));
|
||||
assert_eq!(byte_range_to_columns(line, 0, 1), (0, 8));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn byte_range_display_cols_clamps_past_end() {
|
||||
let line = b"hi";
|
||||
assert_eq!(byte_range_to_display_cols(line, 0, 999), (0, 2));
|
||||
assert_eq!(byte_range_to_columns(line, 0, 999), (0, 2));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -70,6 +70,7 @@ pub mod daemon_attach;
|
|||
pub mod definition;
|
||||
pub mod desktop;
|
||||
pub mod diag;
|
||||
pub mod display_width;
|
||||
pub mod document_highlight;
|
||||
pub mod editor;
|
||||
pub mod editor_core;
|
||||
|
|
|
|||
|
|
@ -43,10 +43,9 @@
|
|||
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use unicode_width::UnicodeWidthChar;
|
||||
|
||||
use crate::buffer::Buffer;
|
||||
use crate::cell::{Cell, CellCoord, CellGrid, Style};
|
||||
use crate::display_width::byte_range_to_columns;
|
||||
use crate::rope::Edit;
|
||||
use crate::view::{View, Viewport};
|
||||
|
||||
|
|
@ -366,30 +365,6 @@ fn line_end(buf: &Buffer, line_offsets: &[u64], line: usize) -> u64 {
|
|||
}
|
||||
}
|
||||
|
||||
fn display_col_for_range(buf: &Buffer, start: u64, end: u64) -> u32 {
|
||||
if end <= start {
|
||||
return 0;
|
||||
}
|
||||
let mut bytes = vec![0u8; (end - start) as usize];
|
||||
buf.snapshot_rope().slice(start, end, &mut bytes);
|
||||
while !bytes.is_empty() && std::str::from_utf8(&bytes).is_err() {
|
||||
bytes.pop();
|
||||
}
|
||||
let Ok(s) = std::str::from_utf8(&bytes) else {
|
||||
return 0;
|
||||
};
|
||||
let mut col = 0;
|
||||
for ch in s.chars() {
|
||||
let width = if ch == '\t' {
|
||||
8 - (col % 8)
|
||||
} else {
|
||||
UnicodeWidthChar::width(ch).unwrap_or(0) as u32
|
||||
};
|
||||
col += width;
|
||||
}
|
||||
col
|
||||
}
|
||||
|
||||
fn render_buffer_style_span(
|
||||
buf: &Buffer,
|
||||
line_offsets: &[u64],
|
||||
|
|
@ -418,8 +393,14 @@ fn render_buffer_style_span(
|
|||
if style_start >= style_end {
|
||||
continue;
|
||||
}
|
||||
let start_col = display_col_for_range(buf, line_start, style_start);
|
||||
let end_col = display_col_for_range(buf, line_start, style_end);
|
||||
let mut line_prefix = vec![0; (style_end - line_start) as usize];
|
||||
buf.snapshot_rope()
|
||||
.slice(line_start, style_end, &mut line_prefix);
|
||||
let (start_col, end_col) = byte_range_to_columns(
|
||||
&line_prefix,
|
||||
(style_start - line_start) as usize,
|
||||
line_prefix.len(),
|
||||
);
|
||||
let start_col = start_col.min(viewport.cell_size.cols);
|
||||
let end_col = end_col.min(viewport.cell_size.cols);
|
||||
for col in start_col..end_col {
|
||||
|
|
|
|||
|
|
@ -21,6 +21,7 @@ use std::sync::{Arc, Mutex};
|
|||
use pmacs_protocol::ByteRange;
|
||||
|
||||
use crate::buffer::BufferId;
|
||||
use crate::display_width::byte_range_to_columns;
|
||||
|
||||
/// One buffer's search state: the resolved query, its matches (byte
|
||||
/// ranges, ascending and non-overlapping), and the active index.
|
||||
|
|
@ -516,7 +517,7 @@ impl View for SearchView {
|
|||
let within_start = (paint_start - line_start) as usize;
|
||||
let within_end = (paint_end - line_start) as usize;
|
||||
let (start_col, end_col) =
|
||||
crate::diag::byte_range_to_display_cols(line_bytes, within_start, within_end);
|
||||
byte_range_to_columns(line_bytes, within_start, within_end);
|
||||
if end_col <= start_col {
|
||||
continue;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2188,6 +2188,7 @@ fn scoped_style_spans(state: &EditorState, vp: &DeclaredViewport) -> Vec<StyleSp
|
|||
query,
|
||||
&layer.tree,
|
||||
source,
|
||||
layer.local_facts.as_deref(),
|
||||
Some(vis_start as usize..vis_end as usize),
|
||||
);
|
||||
for (order, hs) in highlights.iter().enumerate() {
|
||||
|
|
@ -4142,15 +4143,17 @@ mod tests {
|
|||
sid
|
||||
}
|
||||
|
||||
fn seed_rust_parse_view(
|
||||
fn seed_parse_view(
|
||||
state: &EditorState,
|
||||
buffer_id: BufferId,
|
||||
text: &[u8],
|
||||
language_name: &str,
|
||||
path: &str,
|
||||
) -> crate::syntax::ParseViewHandle {
|
||||
let language = state
|
||||
.syntax_registry
|
||||
.language("rust")
|
||||
.expect("rust language");
|
||||
.language(language_name)
|
||||
.unwrap_or_else(|| panic!("{language_name} language"));
|
||||
let mut core = state.core.borrow_mut();
|
||||
let registry_handle = core.registry.clone();
|
||||
let mut registry = registry_handle.borrow_mut();
|
||||
|
|
@ -4160,23 +4163,31 @@ mod tests {
|
|||
pos: 0,
|
||||
bytes: text,
|
||||
})
|
||||
.expect("seed rust text");
|
||||
.expect("seed syntax text");
|
||||
}
|
||||
let parse_view = crate::syntax::ParseView::new(buf, language, "rust".to_owned());
|
||||
let parse_view = crate::syntax::ParseView::new(buf, language, language_name.to_owned());
|
||||
let handle = parse_view.handle();
|
||||
let req = handle.make_request();
|
||||
let bundle = crate::syntax::run_parse(req).expect("initial rust parse");
|
||||
// Mirror the production settle path: resolve each layer's highlight
|
||||
// query before install so the producer can style it (framing Q#IJ2).
|
||||
let bundle = crate::syntax::run_parse(req).expect("initial syntax parse");
|
||||
// Mirror the production settle path: queries and lexical facts travel
|
||||
// with the same bundle the producer reads.
|
||||
handle.install(state.syntax_registry.resolve_layer_queries(&bundle));
|
||||
buf.attach_view(Box::new(parse_view));
|
||||
drop(registry);
|
||||
core.set_buffer_path(buffer_id, Some(std::path::PathBuf::from("/tmp/x.rs")));
|
||||
core.set_buffer_path(buffer_id, Some(std::path::PathBuf::from(path)));
|
||||
drop(core);
|
||||
state.syntax_registry.attach_view(buffer_id, handle.clone());
|
||||
handle
|
||||
}
|
||||
|
||||
fn seed_rust_parse_view(
|
||||
state: &EditorState,
|
||||
buffer_id: BufferId,
|
||||
text: &[u8],
|
||||
) -> crate::syntax::ParseViewHandle {
|
||||
seed_parse_view(state, buffer_id, text, "rust", "/tmp/x.rs")
|
||||
}
|
||||
|
||||
fn seed_markdown_parse_view(
|
||||
state: &EditorState,
|
||||
buffer_id: BufferId,
|
||||
|
|
@ -4261,6 +4272,57 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn viewport_style_producer_uses_settled_local_facts() {
|
||||
let state = empty_state();
|
||||
let buffer_id = active_buffer(&state);
|
||||
let source = b"console;\nfunction f(console) { console; }\n";
|
||||
seed_parse_view(&state, buffer_id, source, "javascript", "/tmp/locals.js");
|
||||
state.syntax_registry.theme().lock().expect("theme").insert(
|
||||
"variable.builtin",
|
||||
Style {
|
||||
fg: crate::cell::Color::Indexed(6),
|
||||
..Style::default()
|
||||
},
|
||||
);
|
||||
|
||||
let style_at = |visible: ByteRange, offset: u64| {
|
||||
scoped_style_spans(
|
||||
&state,
|
||||
&DeclaredViewport {
|
||||
buffer_id,
|
||||
visible,
|
||||
frontend_generation: 0,
|
||||
},
|
||||
)
|
||||
.into_iter()
|
||||
.find(|span| span.range.start <= offset && offset < span.range.end)
|
||||
.map_or_else(Style::default, |span| span.style)
|
||||
};
|
||||
|
||||
assert_eq!(
|
||||
style_at(ByteRange { start: 0, end: 7 }, 0).fg,
|
||||
crate::cell::Color::Indexed(6),
|
||||
"unresolved outer `console` receives the builtin capture"
|
||||
);
|
||||
let inner = source
|
||||
.windows("console".len())
|
||||
.rposition(|window| window == b"console")
|
||||
.expect("inner console") as u64;
|
||||
assert_ne!(
|
||||
style_at(
|
||||
ByteRange {
|
||||
start: inner,
|
||||
end: inner + "console".len() as u64,
|
||||
},
|
||||
inner,
|
||||
)
|
||||
.fg,
|
||||
crate::cell::Color::Indexed(6),
|
||||
"viewport-only highlighting still sees the parameter definition outside the viewport"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn full_buffer_summary_flatten_scales_on_large_grammar_file() {
|
||||
// Perf gate (round-1 finding 1): the file-style summary runs the
|
||||
|
|
|
|||
645
src/syntax.rs
645
src/syntax.rs
|
|
@ -115,6 +115,14 @@ pub struct ParseTreeBundle {
|
|||
pub injection_capped: bool,
|
||||
}
|
||||
|
||||
/// Lexically-local identifier ranges derived from a grammar's bundled
|
||||
/// `locals.scm` query. Ranges are sorted and deduplicated so highlight
|
||||
/// predicate checks are allocation-free binary searches.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct LocalFacts {
|
||||
ranges: Box<[(u32, u32)]>,
|
||||
}
|
||||
|
||||
/// One injection layer within a [`ParseTreeBundle`] (framing Q#IJ1). A
|
||||
/// layer pairs a parse tree with the language that produced it and the
|
||||
/// injection-nesting depth (root = 0). `highlight_query` is resolved on
|
||||
|
|
@ -133,6 +141,10 @@ pub struct Layer {
|
|||
/// `None` when the language ships no highlights, or on the worker
|
||||
/// (pre-settle). Producers read it to style this layer.
|
||||
pub highlight_query: Option<Arc<tree_sitter::Query>>,
|
||||
/// Lexically-local definitions and resolved references for this tree.
|
||||
/// Present only when the highlight query asks about the `local`
|
||||
/// property; computed once when the bundle settles.
|
||||
pub local_facts: Option<Arc<LocalFacts>>,
|
||||
}
|
||||
|
||||
impl ParseTreeBundle {
|
||||
|
|
@ -184,6 +196,7 @@ pub fn run_parse(req: ParseRequest) -> Result<ParseTreeBundle, String> {
|
|||
tree: root_tree,
|
||||
depth: 0,
|
||||
highlight_query: None,
|
||||
local_facts: None,
|
||||
}];
|
||||
let injection_capped =
|
||||
build_injection_layers(&mut layers, req.source.as_ref(), &req.injection_aliases);
|
||||
|
|
@ -309,6 +322,7 @@ fn build_injection_layers(
|
|||
tree,
|
||||
depth: depth + 1,
|
||||
highlight_query: None,
|
||||
local_facts: None,
|
||||
},
|
||||
ranges,
|
||||
));
|
||||
|
|
@ -773,6 +787,10 @@ pub struct LanguageEntry {
|
|||
/// slice (or all-empty fragments) means no highlights: the view
|
||||
/// runs but emits nothing.
|
||||
pub highlights_query: &'static [&'static str],
|
||||
/// Bundled `locals.scm` query fragments, composed base-first like
|
||||
/// [`Self::highlights_query`]. The query supplies lexical scopes,
|
||||
/// definitions, values, and references for `local` property predicates.
|
||||
pub locals_query: &'static [&'static str],
|
||||
/// Bundled `injections.scm` fragments (framing Q#IJ2), joined with a
|
||||
/// newline and compiled on the parse worker to find embedded-language
|
||||
/// regions. Empty for the many grammars that ship none (or don't
|
||||
|
|
@ -801,6 +819,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["rs"],
|
||||
loader: || tree_sitter_rust::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_rust::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[tree_sitter_rust::INJECTIONS_QUERY],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -808,6 +827,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["lua"],
|
||||
loader: || tree_sitter_lua::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_lua::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[tree_sitter_lua::LOCALS_QUERY],
|
||||
injections_query: &[],
|
||||
},
|
||||
// T M9.7: markdown block grammar (`tree_sitter_md::LANGUAGE`) — headers,
|
||||
|
|
@ -824,6 +844,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["md", "markdown"],
|
||||
loader: || tree_sitter_md::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_md::HIGHLIGHT_QUERY_BLOCK],
|
||||
locals_query: &[],
|
||||
injections_query: &[tree_sitter_md::INJECTION_QUERY_BLOCK],
|
||||
},
|
||||
// markdown_inline (framing Q#IJ10) — the inline grammar the block
|
||||
|
|
@ -837,6 +858,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &[],
|
||||
loader: || tree_sitter_md::INLINE_LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_md::HIGHLIGHT_QUERY_INLINE],
|
||||
locals_query: &[],
|
||||
injections_query: &[tree_sitter_md::INJECTION_QUERY_INLINE],
|
||||
},
|
||||
// T M_B3 — C / C++. Lexical highlighting (keywords / strings /
|
||||
|
|
@ -859,6 +881,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["c", "h"],
|
||||
loader: || tree_sitter_c::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_c::HIGHLIGHT_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -866,6 +889,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["cpp", "cc", "cxx", "hpp", "hh", "hxx", "ipp", "inl", "cppm"],
|
||||
loader: || tree_sitter_cpp::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_cpp::HIGHLIGHT_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
// CUDA (`.cu` source, `.cuh` header). A dedicated grammar rather
|
||||
|
|
@ -897,6 +921,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
tree_sitter_cpp::HIGHLIGHT_QUERY,
|
||||
tree_sitter_cuda::HIGHLIGHTS_QUERY,
|
||||
],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
// Shell / bash. Lexical highlighting for the shell family; the LSP
|
||||
|
|
@ -915,6 +940,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["sh", "bash", "zsh", "ksh", "ash", "bats"],
|
||||
loader: || tree_sitter_bash::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_bash::HIGHLIGHT_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
// Filename-identified languages. These files usually have no useful
|
||||
|
|
@ -931,6 +957,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["dockerfile", "containerfile"],
|
||||
loader: || tree_sitter_containerfile::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_containerfile::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -938,6 +965,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["mk", "make"],
|
||||
loader: || tree_sitter_make::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_make::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -945,6 +973,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["cmake"],
|
||||
loader: || tree_sitter_cmake::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_cmake::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
// Grammar-gap languages — these already had LSP configs but no
|
||||
|
|
@ -958,6 +987,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["py", "pyi"],
|
||||
loader: || tree_sitter_python::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_python::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -965,6 +995,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["go"],
|
||||
loader: || tree_sitter_go::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_go::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
// JavaScript / TypeScript. One `tree-sitter-javascript` grammar parses
|
||||
|
|
@ -980,6 +1011,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["js", "mjs", "cjs"],
|
||||
loader: || tree_sitter_javascript::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_javascript::HIGHLIGHT_QUERY],
|
||||
locals_query: &[tree_sitter_javascript::LOCALS_QUERY],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -990,6 +1022,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
tree_sitter_javascript::HIGHLIGHT_QUERY,
|
||||
tree_sitter_javascript::JSX_HIGHLIGHT_QUERY,
|
||||
],
|
||||
locals_query: &[tree_sitter_javascript::LOCALS_QUERY],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -1000,6 +1033,10 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
tree_sitter_javascript::HIGHLIGHT_QUERY,
|
||||
tree_sitter_typescript::HIGHLIGHTS_QUERY,
|
||||
],
|
||||
locals_query: &[
|
||||
tree_sitter_javascript::LOCALS_QUERY,
|
||||
tree_sitter_typescript::LOCALS_QUERY,
|
||||
],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -1011,6 +1048,10 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
tree_sitter_javascript::JSX_HIGHLIGHT_QUERY,
|
||||
tree_sitter_typescript::HIGHLIGHTS_QUERY,
|
||||
],
|
||||
locals_query: &[
|
||||
tree_sitter_javascript::LOCALS_QUERY,
|
||||
tree_sitter_typescript::LOCALS_QUERY,
|
||||
],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -1018,6 +1059,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["toml"],
|
||||
loader: || tree_sitter_toml_ng::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_toml_ng::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -1025,6 +1067,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["zig", "zon"],
|
||||
loader: || tree_sitter_zig::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_zig::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
// JSON + YAML — config formats, both self-contained highlights and no
|
||||
|
|
@ -1039,6 +1082,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["json"],
|
||||
loader: || tree_sitter_json::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_json::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
LanguageEntry {
|
||||
|
|
@ -1046,6 +1090,7 @@ pub const BUILTIN_LANGUAGES: &[LanguageEntry] = &[
|
|||
extensions: &["yaml", "yml"],
|
||||
loader: || tree_sitter_yaml::LANGUAGE.into(),
|
||||
highlights_query: &[tree_sitter_yaml::HIGHLIGHTS_QUERY],
|
||||
locals_query: &[],
|
||||
injections_query: &[],
|
||||
},
|
||||
];
|
||||
|
|
@ -1076,6 +1121,9 @@ pub struct SyntaxRegistry {
|
|||
/// compilation failure (e.g. grammar / query ABI skew) is
|
||||
/// cached as `Err(message)` so we don't burn cycles re-trying.
|
||||
queries: RefCell<HashMap<String, Result<Arc<tree_sitter::Query>, String>>>,
|
||||
/// Compiled `locals.scm` query per language. Like `queries`, both
|
||||
/// compilation failures and absent query sources are cached.
|
||||
local_queries: RefCell<HashMap<String, Result<Arc<tree_sitter::Query>, String>>>,
|
||||
/// Fence-name → canonical-language alias map (framing Q#IJ4). Seeded
|
||||
/// with [`default_injection_aliases`]; Lua adds to it through
|
||||
/// [`Self::register_injection_alias`]. Snapshotted into each
|
||||
|
|
@ -1107,6 +1155,7 @@ impl SyntaxRegistry {
|
|||
parse_jobs: RefCell::new(HashMap::new()),
|
||||
extra_extensions: RefCell::new(HashMap::new()),
|
||||
queries: RefCell::new(HashMap::new()),
|
||||
local_queries: RefCell::new(HashMap::new()),
|
||||
injection_aliases: RefCell::new(default_injection_aliases()),
|
||||
theme: Arc::new(Mutex::new(Theme::default_dark())),
|
||||
}
|
||||
|
|
@ -1273,6 +1322,32 @@ impl SyntaxRegistry {
|
|||
result
|
||||
}
|
||||
|
||||
/// Lazy-compile and cache the bundled `locals.scm` query for
|
||||
/// `lang_name`. Empty sources and compilation failures are cached.
|
||||
#[must_use]
|
||||
pub fn locals_query(&self, lang_name: &str) -> Option<Arc<tree_sitter::Query>> {
|
||||
if let Some(slot) = self.local_queries.borrow().get(lang_name) {
|
||||
return slot.as_ref().ok().cloned();
|
||||
}
|
||||
let language = self.language(lang_name)?;
|
||||
let entry = BUILTIN_LANGUAGES.iter().find(|e| e.name == lang_name);
|
||||
let source = entry.map_or_else(String::new, |e| e.locals_query.join("\n"));
|
||||
if source.trim().is_empty() {
|
||||
self.local_queries
|
||||
.borrow_mut()
|
||||
.insert(lang_name.to_owned(), Err("no locals query".to_owned()));
|
||||
return None;
|
||||
}
|
||||
let compiled = tree_sitter::Query::new(&language, &source)
|
||||
.map(Arc::new)
|
||||
.map_err(|e| format!("compile {lang_name} locals: {e:?}"));
|
||||
let result = compiled.as_ref().ok().cloned();
|
||||
self.local_queries
|
||||
.borrow_mut()
|
||||
.insert(lang_name.to_owned(), compiled);
|
||||
result
|
||||
}
|
||||
|
||||
/// Add or override a fence-name → language alias (framing Q#IJ4). The
|
||||
/// alias key is case-folded to match the resolver. Called from Lua via
|
||||
/// `pmacs.parse.injection_aliases`.
|
||||
|
|
@ -1299,11 +1374,26 @@ impl SyntaxRegistry {
|
|||
let layers = raw
|
||||
.layers
|
||||
.iter()
|
||||
.map(|l| Layer {
|
||||
language_name: l.language_name.clone(),
|
||||
tree: l.tree.clone(),
|
||||
depth: l.depth,
|
||||
highlight_query: self.highlights_query(&l.language_name),
|
||||
.map(|layer| {
|
||||
let highlight_query = self.highlights_query(&layer.language_name);
|
||||
let local_facts = highlight_query
|
||||
.as_deref()
|
||||
.filter(|query| query_uses_local_predicates(query))
|
||||
.and_then(|_| self.locals_query(&layer.language_name))
|
||||
.map(|query| {
|
||||
Arc::new(compute_local_facts(
|
||||
&query,
|
||||
&layer.tree,
|
||||
raw.source.as_ref(),
|
||||
))
|
||||
});
|
||||
Layer {
|
||||
language_name: layer.language_name.clone(),
|
||||
tree: layer.tree.clone(),
|
||||
depth: layer.depth,
|
||||
highlight_query,
|
||||
local_facts,
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
Arc::new(ParseTreeBundle {
|
||||
|
|
@ -1316,6 +1406,148 @@ impl SyntaxRegistry {
|
|||
}
|
||||
}
|
||||
|
||||
fn query_uses_local_predicates(query: &tree_sitter::Query) -> bool {
|
||||
(0..query.pattern_count()).any(|pattern| {
|
||||
query
|
||||
.property_predicates(pattern)
|
||||
.iter()
|
||||
.any(|(property, _)| property.key.as_ref() == "local")
|
||||
})
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct LocalDefinition {
|
||||
name_range: std::ops::Range<usize>,
|
||||
value_range: std::ops::Range<usize>,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct LocalScope {
|
||||
inherits: bool,
|
||||
range: std::ops::Range<usize>,
|
||||
definitions: Vec<LocalDefinition>,
|
||||
}
|
||||
|
||||
impl LocalFacts {
|
||||
fn contains(&self, start_byte: usize, end_byte: usize) -> bool {
|
||||
let (Ok(start_byte), Ok(end_byte)) = (u32::try_from(start_byte), u32::try_from(end_byte))
|
||||
else {
|
||||
return false;
|
||||
};
|
||||
self.ranges.binary_search(&(start_byte, end_byte)).is_ok()
|
||||
}
|
||||
}
|
||||
|
||||
/// Resolve lexical definitions and references according to Tree-sitter's
|
||||
/// standard `locals.scm` capture conventions.
|
||||
fn compute_local_facts(
|
||||
query: &tree_sitter::Query,
|
||||
tree: &tree_sitter::Tree,
|
||||
source: &[u8],
|
||||
) -> LocalFacts {
|
||||
let scope_capture = query.capture_index_for_name("local.scope");
|
||||
let definition_capture = query.capture_index_for_name("local.definition");
|
||||
let value_capture = query.capture_index_for_name("local.definition-value");
|
||||
let reference_capture = query.capture_index_for_name("local.reference");
|
||||
|
||||
let mut scopes = vec![LocalScope {
|
||||
inherits: false,
|
||||
range: 0..source.len(),
|
||||
definitions: Vec::new(),
|
||||
}];
|
||||
let mut ranges = Vec::new();
|
||||
let mut cursor = tree_sitter::QueryCursor::new();
|
||||
let mut captures = cursor.captures(query, tree.root_node(), source);
|
||||
|
||||
while let Some((query_match, capture_index)) = captures.next() {
|
||||
let capture = query_match.captures[*capture_index];
|
||||
let node_range = capture.node.byte_range();
|
||||
while scopes.len() > 1
|
||||
&& node_range.start > scopes.last().expect("root scope exists").range.end
|
||||
{
|
||||
scopes.pop();
|
||||
}
|
||||
|
||||
if Some(capture.index) == scope_capture {
|
||||
let mut inherits = true;
|
||||
for property in query.property_settings(query_match.pattern_index) {
|
||||
if property.key.as_ref() == "local.scope-inherits" {
|
||||
inherits = property
|
||||
.value
|
||||
.as_deref()
|
||||
.is_none_or(|value| value == "true");
|
||||
}
|
||||
}
|
||||
scopes.push(LocalScope {
|
||||
inherits,
|
||||
range: node_range,
|
||||
definitions: Vec::new(),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
if Some(capture.index) == definition_capture {
|
||||
let Some(_) = source.get(node_range.clone()) else {
|
||||
continue;
|
||||
};
|
||||
let value_range = query_match
|
||||
.captures
|
||||
.iter()
|
||||
.find(|candidate| Some(candidate.index) == value_capture)
|
||||
.map_or(0..0, |candidate| candidate.node.byte_range());
|
||||
scopes
|
||||
.last_mut()
|
||||
.expect("root scope exists")
|
||||
.definitions
|
||||
.push(LocalDefinition {
|
||||
name_range: node_range.clone(),
|
||||
value_range,
|
||||
});
|
||||
if let (Ok(start), Ok(end)) = (
|
||||
u32::try_from(node_range.start),
|
||||
u32::try_from(node_range.end),
|
||||
) {
|
||||
ranges.push((start, end));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if Some(capture.index) != reference_capture {
|
||||
continue;
|
||||
}
|
||||
let Some(name) = source.get(node_range.clone()) else {
|
||||
continue;
|
||||
};
|
||||
let mut resolved = false;
|
||||
for scope in scopes.iter().rev() {
|
||||
if scope.definitions.iter().rev().any(|definition| {
|
||||
node_range.start >= definition.value_range.end
|
||||
&& source.get(definition.name_range.clone()) == Some(name)
|
||||
}) {
|
||||
resolved = true;
|
||||
break;
|
||||
}
|
||||
if !scope.inherits {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if resolved
|
||||
&& let (Ok(start), Ok(end)) = (
|
||||
u32::try_from(node_range.start),
|
||||
u32::try_from(node_range.end),
|
||||
)
|
||||
{
|
||||
ranges.push((start, end));
|
||||
}
|
||||
}
|
||||
|
||||
ranges.sort_unstable();
|
||||
ranges.dedup();
|
||||
LocalFacts {
|
||||
ranges: ranges.into_boxed_slice(),
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for SyntaxRegistry {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
|
|
@ -1418,20 +1650,22 @@ pub fn compute_highlight_spans_in_range(
|
|||
query,
|
||||
bundle.root_tree(),
|
||||
bundle.source.as_ref(),
|
||||
bundle.layers[0].local_facts.as_deref(),
|
||||
byte_range,
|
||||
)
|
||||
}
|
||||
|
||||
/// Like [`compute_highlight_spans_in_range`] but over an explicit
|
||||
/// `(tree, source)` — the per-layer form the producers call for each
|
||||
/// injection layer (framing Q#IJ7). `source` is the whole buffer; a
|
||||
/// child layer's tree carries absolute offsets into it, so the same
|
||||
/// `(tree, source, local_facts)` layer tuple — the form producers call for
|
||||
/// each injection layer (framing Q#IJ7 and Q#LQ5). `source` is the whole
|
||||
/// buffer; a child layer's tree carries absolute offsets into it, so the same
|
||||
/// capture walk works unchanged.
|
||||
#[must_use]
|
||||
pub fn compute_highlight_spans_for(
|
||||
query: &tree_sitter::Query,
|
||||
tree: &tree_sitter::Tree,
|
||||
source: &[u8],
|
||||
local_facts: Option<&LocalFacts>,
|
||||
byte_range: Option<std::ops::Range<usize>>,
|
||||
) -> Vec<HighlightSpan> {
|
||||
let mut spans = Vec::new();
|
||||
|
|
@ -1441,30 +1675,33 @@ pub fn compute_highlight_spans_for(
|
|||
}
|
||||
let root = tree.root_node();
|
||||
let mut iter = cursor.captures(query, root, source);
|
||||
while let Some((qmatch, capture_idx)) = iter.next() {
|
||||
// Fail-closed on the locals property predicate. The capture
|
||||
// iterator already applies text predicates (`#eq?`/`#match?`/
|
||||
// `#any-of?`), but `#is? local` / `#is-not? local` are *property*
|
||||
// predicates (`Query::property_predicates`) that need a scope map
|
||||
// built from the grammar's LOCALS_QUERY, which pmacs does not run.
|
||||
// Applying such a capture regardless mis-styles shadowed locals —
|
||||
// e.g. a local `console`/`require` in JS/TS would still capture as
|
||||
// `@variable.builtin`/`@function.builtin`. Until locals processing
|
||||
// exists, drop captures whose pattern carries one; the identifier
|
||||
// falls back to its non-builtin capture. `#set!` (property
|
||||
// *settings*) is a different API and is not consulted here.
|
||||
if query
|
||||
.property_predicates(qmatch.pattern_index)
|
||||
while let Some((query_match, capture_index)) = iter.next() {
|
||||
let capture = query_match.captures[*capture_index];
|
||||
let local_predicates_match = query
|
||||
.property_predicates(query_match.pattern_index)
|
||||
.iter()
|
||||
.any(|(prop, _)| &*prop.key == "local")
|
||||
{
|
||||
.filter(|(property, _)| property.key.as_ref() == "local")
|
||||
.all(|(property, positive)| {
|
||||
let node = property.capture_id.map_or(Some(capture.node), |target| {
|
||||
query_match
|
||||
.captures
|
||||
.iter()
|
||||
.find(|candidate| candidate.index as usize == target)
|
||||
.map(|candidate| candidate.node)
|
||||
});
|
||||
let is_local = node.is_some_and(|node| {
|
||||
local_facts
|
||||
.is_some_and(|facts| facts.contains(node.start_byte(), node.end_byte()))
|
||||
});
|
||||
is_local == *positive
|
||||
});
|
||||
if !local_predicates_match {
|
||||
continue;
|
||||
}
|
||||
let cap = qmatch.captures[*capture_idx];
|
||||
spans.push(HighlightSpan {
|
||||
start_byte: cap.node.start_byte() as u32,
|
||||
end_byte: cap.node.end_byte() as u32,
|
||||
capture_index: cap.index,
|
||||
start_byte: capture.node.start_byte() as u32,
|
||||
end_byte: capture.node.end_byte() as u32,
|
||||
capture_index: capture.index,
|
||||
});
|
||||
}
|
||||
// Wider-first ordering at equal start: later writes (the
|
||||
|
|
@ -1684,7 +1921,13 @@ mod tests {
|
|||
.highlight_query
|
||||
.as_ref()
|
||||
.expect("inline highlights resolved at settle");
|
||||
let spans = compute_highlight_spans_for(hquery, &inline.tree, &bundle.source, None);
|
||||
let spans = compute_highlight_spans_for(
|
||||
hquery,
|
||||
&inline.tree,
|
||||
&bundle.source,
|
||||
inline.local_facts.as_deref(),
|
||||
None,
|
||||
);
|
||||
assert!(
|
||||
!spans.is_empty(),
|
||||
"the inline layer produces highlight spans across both ranges"
|
||||
|
|
@ -2221,7 +2464,13 @@ mod tests {
|
|||
.highlight_query
|
||||
.as_ref()
|
||||
.expect("yaml highlights resolved");
|
||||
let spans = compute_highlight_spans_for(query, &yaml.tree, &bundle.source, None);
|
||||
let spans = compute_highlight_spans_for(
|
||||
query,
|
||||
&yaml.tree,
|
||||
&bundle.source,
|
||||
yaml.local_facts.as_deref(),
|
||||
None,
|
||||
);
|
||||
assert!(!spans.is_empty(), "the yaml frontmatter layer highlights");
|
||||
}
|
||||
|
||||
|
|
@ -2418,54 +2667,296 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn javascript_shadowed_builtin_is_not_mislabeled() {
|
||||
// `#is-not? local` (JS/TS use it for console/require/etc.) needs a
|
||||
// scope map from the LOCALS_QUERY we don't run, so
|
||||
// `compute_highlight_spans` drops captures guarded by it. Here
|
||||
// `console` is a LOCAL declaration — it must not surface as a
|
||||
// `*.builtin` capture (which is what a naive run of the shared JS
|
||||
// query would produce).
|
||||
let reg = SyntaxRegistry::new();
|
||||
let language = reg.language("javascript").expect("javascript loads");
|
||||
let query = reg
|
||||
.highlights_query("javascript")
|
||||
.expect("javascript highlights compile");
|
||||
let mut buf = fresh_buffer("shadow.js");
|
||||
buf.apply_edit(EditOp::Insert {
|
||||
pos: 0,
|
||||
bytes: b"const console = 5;\nconsole;\n",
|
||||
})
|
||||
.unwrap();
|
||||
let view = ParseView::new(&buf, language, "javascript".to_owned());
|
||||
let handle = view.handle();
|
||||
let _vid = buf.attach_view(Box::new(view));
|
||||
let bundle = parse_synchronously(&handle);
|
||||
let spans = compute_highlight_spans(&query, &bundle);
|
||||
assert!(!spans.is_empty(), "the JS query produced highlight spans");
|
||||
let names = query.capture_names();
|
||||
let builtin: Vec<&str> = spans
|
||||
.iter()
|
||||
.map(|s| names[s.capture_index as usize])
|
||||
.filter(|n| n.contains("builtin"))
|
||||
.collect();
|
||||
assert!(
|
||||
builtin.is_empty(),
|
||||
"a locally-shadowed `console` must not get a *.builtin capture; got {builtin:?}"
|
||||
);
|
||||
// ...and dropping the builtin pattern must not strip *all* styling:
|
||||
// each `console` occurrence still keeps its ordinary `@variable`
|
||||
// capture (the fallback), so the loss is only the `.builtin` refine.
|
||||
let src = "const console = 5;\nconsole;\n";
|
||||
for (pos, _) in src.match_indices("console") {
|
||||
let (start, end) = (pos as u32, (pos + "console".len()) as u32);
|
||||
let caps: Vec<&str> = spans
|
||||
.iter()
|
||||
.filter(|s| s.start_byte == start && s.end_byte == end)
|
||||
.map(|s| names[s.capture_index as usize])
|
||||
.collect();
|
||||
fn local_sensitive_builtin_highlights_have_compilable_locals_queries() {
|
||||
let registry = SyntaxRegistry::new();
|
||||
for entry in BUILTIN_LANGUAGES {
|
||||
let Some(highlights) = registry.highlights_query(entry.name) else {
|
||||
continue;
|
||||
};
|
||||
if !query_uses_local_predicates(&highlights) {
|
||||
continue;
|
||||
}
|
||||
assert!(
|
||||
caps.iter().any(|n| n.starts_with("variable")),
|
||||
"`console` at byte {pos} keeps a variable capture; got {caps:?}"
|
||||
entry
|
||||
.locals_query
|
||||
.iter()
|
||||
.any(|fragment| !fragment.trim().is_empty()),
|
||||
"`{}` highlights use a local predicate but ship no locals query",
|
||||
entry.name
|
||||
);
|
||||
assert!(
|
||||
registry.locals_query(entry.name).is_some(),
|
||||
"`{}` highlights use a local predicate but its locals query does not compile",
|
||||
entry.name
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn javascript_local_predicates_distinguish_lexical_scope() {
|
||||
let registry = SyntaxRegistry::new();
|
||||
let source = b"console.log('outer');\n\
|
||||
require('outer');\n\
|
||||
function f(console, require) {\n\
|
||||
console.log('inner');\n\
|
||||
require('inner');\n\
|
||||
}\n\
|
||||
window.alert('outer');\n";
|
||||
let bundle = parse_layered(®istry, "javascript", source);
|
||||
let layer = &bundle.layers[0];
|
||||
let query = layer
|
||||
.highlight_query
|
||||
.as_deref()
|
||||
.expect("javascript highlights compile");
|
||||
assert!(
|
||||
layer.local_facts.is_some(),
|
||||
"a local-sensitive highlight query must settle lexical facts"
|
||||
);
|
||||
let spans = compute_highlight_spans(query, &bundle);
|
||||
let names = query.capture_names();
|
||||
let captures_at = |start: usize, len: usize| -> Vec<&str> {
|
||||
spans
|
||||
.iter()
|
||||
.filter(|span| {
|
||||
span.start_byte == start as u32 && span.end_byte == (start + len) as u32
|
||||
})
|
||||
.map(|span| names[span.capture_index as usize])
|
||||
.collect()
|
||||
};
|
||||
|
||||
for identifier in ["console", "require"] {
|
||||
let positions: Vec<usize> = std::str::from_utf8(source)
|
||||
.expect("fixture is UTF-8")
|
||||
.match_indices(identifier)
|
||||
.map(|(position, _)| position)
|
||||
.collect();
|
||||
assert_eq!(positions.len(), 3, "fixture has three `{identifier}` uses");
|
||||
assert!(
|
||||
captures_at(positions[0], identifier.len())
|
||||
.iter()
|
||||
.any(|name| name.ends_with(".builtin")),
|
||||
"unshadowed outer `{identifier}` keeps its builtin refinement"
|
||||
);
|
||||
for position in &positions[1..] {
|
||||
let captures = captures_at(*position, identifier.len());
|
||||
assert!(
|
||||
!captures.iter().any(|name| name.ends_with(".builtin")),
|
||||
"local `{identifier}` at byte {position} is not builtin: {captures:?}"
|
||||
);
|
||||
assert!(
|
||||
captures.iter().any(|name| name.starts_with("variable")),
|
||||
"local `{identifier}` keeps an ordinary variable capture: {captures:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let window = std::str::from_utf8(source)
|
||||
.expect("fixture is UTF-8")
|
||||
.find("window")
|
||||
.expect("window fixture");
|
||||
assert!(
|
||||
captures_at(window, "window".len())
|
||||
.iter()
|
||||
.any(|name| name == &"variable.builtin"),
|
||||
"an unresolved builtin after the function remains builtin"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn positive_and_capture_qualified_local_predicates_use_resolved_facts() {
|
||||
let registry = SyntaxRegistry::new();
|
||||
let source = b"let f = () => {};\nf();\ng();\n";
|
||||
let bundle = parse_layered(®istry, "javascript", source);
|
||||
let language = registry.language("javascript").expect("javascript loads");
|
||||
let facts = bundle.layers[0]
|
||||
.local_facts
|
||||
.as_deref()
|
||||
.expect("javascript local facts settle");
|
||||
|
||||
let positive = tree_sitter::Query::new(&language, "((identifier) @local-id (#is? local))")
|
||||
.expect("positive local predicate compiles");
|
||||
let positive_spans =
|
||||
compute_highlight_spans_for(&positive, bundle.root_tree(), source, Some(facts), None);
|
||||
let f_positions: Vec<usize> = std::str::from_utf8(source)
|
||||
.expect("fixture is UTF-8")
|
||||
.match_indices('f')
|
||||
.map(|(position, _)| position)
|
||||
.collect();
|
||||
assert_eq!(f_positions.len(), 2);
|
||||
for position in f_positions {
|
||||
assert!(
|
||||
positive_spans.iter().any(|span| {
|
||||
span.start_byte == position as u32 && span.end_byte == (position + 1) as u32
|
||||
}),
|
||||
"definition/reference `f` at byte {position} is local"
|
||||
);
|
||||
}
|
||||
let g_position = std::str::from_utf8(source)
|
||||
.expect("fixture is UTF-8")
|
||||
.find("g()")
|
||||
.expect("g call");
|
||||
assert!(
|
||||
positive_spans
|
||||
.iter()
|
||||
.all(|span| span.start_byte != g_position as u32),
|
||||
"unresolved `g` does not satisfy #is? local"
|
||||
);
|
||||
|
||||
let qualified = tree_sitter::Query::new(
|
||||
&language,
|
||||
"((call_expression function: (identifier) @callee) @call \
|
||||
(#is? @callee local))",
|
||||
)
|
||||
.expect("capture-qualified local predicate compiles");
|
||||
let qualified_spans =
|
||||
compute_highlight_spans_for(&qualified, bundle.root_tree(), source, Some(facts), None);
|
||||
let qualified_names = qualified.capture_names();
|
||||
assert!(
|
||||
qualified_spans.iter().any(|span| {
|
||||
qualified_names[span.capture_index as usize] == "call"
|
||||
&& span.start_byte
|
||||
== source
|
||||
.windows(4)
|
||||
.position(|window| window == b"f();")
|
||||
.expect("f call") as u32
|
||||
}),
|
||||
"the call whose @callee is local satisfies the qualified predicate"
|
||||
);
|
||||
assert!(
|
||||
qualified_spans
|
||||
.iter()
|
||||
.all(|span| span.start_byte != g_position as u32),
|
||||
"the call whose @callee is unresolved fails the qualified predicate"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn local_definition_value_and_scope_inheritance_control_resolution() {
|
||||
let registry = SyntaxRegistry::new();
|
||||
let language = registry.language("javascript").expect("javascript loads");
|
||||
|
||||
let value_source = b"let x = x;\nx;\n";
|
||||
let value_tree = {
|
||||
let mut parser = tree_sitter::Parser::new();
|
||||
parser
|
||||
.set_language(&language)
|
||||
.expect("set javascript language");
|
||||
parser
|
||||
.parse(value_source, None)
|
||||
.expect("parse value fixture")
|
||||
};
|
||||
let value_locals = tree_sitter::Query::new(
|
||||
&language,
|
||||
"(variable_declarator \
|
||||
name: (identifier) @local.definition \
|
||||
value: (identifier) @local.definition-value) \
|
||||
(identifier) @local.reference",
|
||||
)
|
||||
.expect("definition-value locals query compiles");
|
||||
let value_facts = compute_local_facts(&value_locals, &value_tree, value_source);
|
||||
let x_positions: Vec<usize> = std::str::from_utf8(value_source)
|
||||
.expect("fixture is UTF-8")
|
||||
.match_indices('x')
|
||||
.map(|(position, _)| position)
|
||||
.collect();
|
||||
assert_eq!(x_positions.len(), 3);
|
||||
assert!(value_facts.contains(x_positions[0], x_positions[0] + 1));
|
||||
assert!(
|
||||
!value_facts.contains(x_positions[1], x_positions[1] + 1),
|
||||
"a definition is not visible inside its own value"
|
||||
);
|
||||
assert!(value_facts.contains(x_positions[2], x_positions[2] + 1));
|
||||
|
||||
let scope_source = b"let x = 1;\nfunction f() { x; }\nx;\n";
|
||||
let scope_tree = {
|
||||
let mut parser = tree_sitter::Parser::new();
|
||||
parser
|
||||
.set_language(&language)
|
||||
.expect("set javascript language");
|
||||
parser
|
||||
.parse(scope_source, None)
|
||||
.expect("parse scope fixture")
|
||||
};
|
||||
let scope_locals = tree_sitter::Query::new(
|
||||
&language,
|
||||
"((function_declaration) @local.scope \
|
||||
(#set! local.scope-inherits false)) \
|
||||
(variable_declarator name: (identifier) @local.definition) \
|
||||
(identifier) @local.reference",
|
||||
)
|
||||
.expect("non-inheriting locals query compiles");
|
||||
let scope_facts = compute_local_facts(&scope_locals, &scope_tree, scope_source);
|
||||
let x_positions: Vec<usize> = std::str::from_utf8(scope_source)
|
||||
.expect("fixture is UTF-8")
|
||||
.match_indices('x')
|
||||
.map(|(position, _)| position)
|
||||
.collect();
|
||||
assert_eq!(x_positions.len(), 3);
|
||||
assert!(scope_facts.contains(x_positions[0], x_positions[0] + 1));
|
||||
assert!(
|
||||
!scope_facts.contains(x_positions[1], x_positions[1] + 1),
|
||||
"a non-inheriting scope cannot see the outer `x`"
|
||||
);
|
||||
assert!(
|
||||
scope_facts.contains(x_positions[2], x_positions[2] + 1),
|
||||
"leaving the scope restores outer resolution"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn typescript_locals_compose_javascript_scopes_and_parameter_delta() {
|
||||
let registry = SyntaxRegistry::new();
|
||||
for (language_name, source) in [
|
||||
(
|
||||
"typescript",
|
||||
&b"function f(console: string) { console.log('x'); }\n\
|
||||
window.alert('x');\n"[..],
|
||||
),
|
||||
(
|
||||
"typescriptreact",
|
||||
&b"function F(console: string) { return <div>{console}</div>; }\n\
|
||||
window.alert('x');\n"[..],
|
||||
),
|
||||
] {
|
||||
let locals = registry
|
||||
.locals_query(language_name)
|
||||
.unwrap_or_else(|| panic!("{language_name} locals compile"));
|
||||
assert!(
|
||||
locals.capture_index_for_name("local.scope").is_some()
|
||||
&& locals.capture_index_for_name("local.definition").is_some()
|
||||
&& locals.capture_index_for_name("local.reference").is_some(),
|
||||
"{language_name} includes JavaScript's scopes and references"
|
||||
);
|
||||
|
||||
let bundle = parse_layered(®istry, language_name, source);
|
||||
let layer = &bundle.layers[0];
|
||||
let query = layer
|
||||
.highlight_query
|
||||
.as_deref()
|
||||
.expect("highlights compile");
|
||||
let spans = compute_highlight_spans(query, &bundle);
|
||||
let names = query.capture_names();
|
||||
let text = std::str::from_utf8(source).expect("fixture is UTF-8");
|
||||
for (position, _) in text.match_indices("console") {
|
||||
assert!(
|
||||
spans
|
||||
.iter()
|
||||
.filter(|span| {
|
||||
span.start_byte == position as u32
|
||||
&& span.end_byte == (position + "console".len()) as u32
|
||||
})
|
||||
.all(|span| !names[span.capture_index as usize].ends_with(".builtin")),
|
||||
"{language_name} parameter/reference `console` is local"
|
||||
);
|
||||
}
|
||||
let window = text.find("window").expect("window fixture");
|
||||
assert!(
|
||||
spans.iter().any(|span| {
|
||||
span.start_byte == window as u32
|
||||
&& span.end_byte == (window + "window".len()) as u32
|
||||
&& names[span.capture_index as usize] == "variable.builtin"
|
||||
}),
|
||||
"{language_name} unresolved `window` remains builtin"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -19,10 +19,9 @@
|
|||
//! Main thread only. The view is held inside a [`Buffer`], which is itself
|
||||
//! main-only.
|
||||
|
||||
use unicode_width::UnicodeWidthChar;
|
||||
|
||||
use crate::buffer::{Buffer, BufferError};
|
||||
use crate::cell::{Cell, CellCoord, CellGrid, Glyph, Style};
|
||||
use crate::display_width::{advance_char, valid_prefix_width};
|
||||
use crate::rope::{Edit, Position};
|
||||
use crate::view::{DisplayCoord, View, Viewport};
|
||||
|
||||
|
|
@ -30,28 +29,10 @@ use crate::view::{DisplayCoord, View, Viewport};
|
|||
// Tuning
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Tab stop width in display columns. A `\t` advances to the next column
|
||||
/// that is a multiple of this value.
|
||||
const TAB_WIDTH: u32 = 8;
|
||||
|
||||
/// Line-prefix lengths up to this many bytes are decoded on the stack in
|
||||
/// [`TextView::pos_to_display`]; longer prefixes fall back to a heap buffer.
|
||||
const STACK_CAP: usize = 256;
|
||||
|
||||
/// Display width of `ch` when drawn starting at column `current_col`.
|
||||
///
|
||||
/// Tabs expand to the next [`TAB_WIDTH`]-aligned column, so they need the
|
||||
/// running column to compute width. Everything else delegates to
|
||||
/// [`UnicodeWidthChar`]: control characters return 0 (skipped by the
|
||||
/// caller), printable characters return 1, wide characters return 2.
|
||||
fn char_display_width(ch: char, current_col: u32) -> u32 {
|
||||
if ch == '\t' {
|
||||
TAB_WIDTH - (current_col % TAB_WIDTH)
|
||||
} else {
|
||||
UnicodeWidthChar::width(ch).unwrap_or(0) as u32
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// TextView
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
@ -196,19 +177,7 @@ impl View for TextView {
|
|||
&mut heap_buf
|
||||
};
|
||||
buf.snapshot_rope().slice(line_start, pos, bytes);
|
||||
// If `pos` fell inside a multi-byte codepoint, keep only the bytes up to
|
||||
// the last complete codepoint. `valid_up_to()` gives that boundary in
|
||||
// one step, replacing the old pop-one-byte-and-revalidate loop. (Only
|
||||
// trailing bytes can be invalid here, since the slice is a prefix of
|
||||
// valid UTF-8 cut at `pos`.)
|
||||
let s = match std::str::from_utf8(bytes) {
|
||||
Ok(valid) => valid,
|
||||
Err(e) => std::str::from_utf8(&bytes[..e.valid_up_to()]).unwrap(),
|
||||
};
|
||||
let mut col: u32 = 0;
|
||||
for ch in s.chars() {
|
||||
col += char_display_width(ch, col);
|
||||
}
|
||||
let col = valid_prefix_width(bytes);
|
||||
Some(DisplayCoord::new(row_idx as u32, col))
|
||||
}
|
||||
|
||||
|
|
@ -228,7 +197,7 @@ impl View for TextView {
|
|||
walked_bytes = byte_idx;
|
||||
return Some(line_start + walked_bytes as u64);
|
||||
}
|
||||
walked_cols += char_display_width(ch, walked_cols);
|
||||
walked_cols = advance_char(walked_cols, ch);
|
||||
walked_bytes = byte_idx + ch.len_utf8();
|
||||
}
|
||||
// Past the line's last codepoint: clamp to the line's visible end.
|
||||
|
|
@ -265,8 +234,8 @@ impl View for TextView {
|
|||
break;
|
||||
}
|
||||
if ch == '\t' {
|
||||
// Expand to the next TAB_WIDTH-aligned column with spaces.
|
||||
let pad = char_display_width(ch, col);
|
||||
// Expand to the next protocol-wide tab stop with spaces.
|
||||
let pad = advance_char(col, ch) - col;
|
||||
for _ in 0..pad {
|
||||
if col >= max_cols {
|
||||
break;
|
||||
|
|
@ -279,7 +248,7 @@ impl View for TextView {
|
|||
}
|
||||
continue;
|
||||
}
|
||||
let width = UnicodeWidthChar::width(ch).unwrap_or(0) as u32;
|
||||
let width = advance_char(col, ch) - col;
|
||||
if width == 0 {
|
||||
// Combining mark or other zero-width control: M1.5
|
||||
// skips; M2+ will attach to the previous cell as
|
||||
|
|
@ -531,7 +500,7 @@ mod tests {
|
|||
|
||||
#[test]
|
||||
fn tab_aligned_input_advances_full_width() {
|
||||
// 8 chars then tab: tab pads from col 8 to col 16 (a full TAB_WIDTH).
|
||||
// 8 chars then tab: the protocol tab stop advances col 8 to col 16.
|
||||
let (buf, view) = attached(b"01234567\tx");
|
||||
assert_eq!(view.pos_to_display(&buf, 8), Some(DisplayCoord::new(0, 8)));
|
||||
assert_eq!(view.pos_to_display(&buf, 9), Some(DisplayCoord::new(0, 16)));
|
||||
|
|
|
|||
|
|
@ -771,6 +771,98 @@ fn m4_3_theming_via_lua_color_scheme() {
|
|||
);
|
||||
}
|
||||
|
||||
/// Locals-query acceptance: the shipped JavaScript grammar must distinguish an
|
||||
/// unresolved builtin from a lexically-shadowed parameter, then replace the
|
||||
/// classification with the fresh parse bundle after an edit removes the
|
||||
/// shadow. The theme maps only `variable.builtin`, making the classification
|
||||
/// observable in rendered cells rather than through an internal scope map.
|
||||
#[test]
|
||||
fn m4_locals_query_shadowing_and_edit_freshness() {
|
||||
use pmacs::cell::Color;
|
||||
|
||||
const COLS: usize = 40;
|
||||
const BUILTIN: Color = Color::Indexed(6);
|
||||
let initial = "console;\nfunction f(console) {\n console;\n}\n";
|
||||
let edited = "console;\nfunction f(logger) {\n console;\n}\n";
|
||||
let dir = tempfile::tempdir().expect("tempdir");
|
||||
let path = dir.path().join("locals.js");
|
||||
std::fs::write(&path, initial.as_bytes()).expect("write JavaScript fixture");
|
||||
|
||||
let mut state = open_and_wait_for_parse(path);
|
||||
state
|
||||
.lua_host
|
||||
.lua()
|
||||
.load(
|
||||
r#"
|
||||
pmacs.theme.set {
|
||||
["variable.builtin"] = { fg = 6, bold = true },
|
||||
}
|
||||
"#,
|
||||
)
|
||||
.exec()
|
||||
.expect("apply builtin-only theme");
|
||||
|
||||
let before = render_active_window_to_grid(&mut state, 5, COLS as u32);
|
||||
for (col, cell) in before.iter().take(7).enumerate() {
|
||||
assert_eq!(
|
||||
cell.style.fg, BUILTIN,
|
||||
"unshadowed row-0 `console` byte {col} is builtin"
|
||||
);
|
||||
assert!(
|
||||
cell.style.bold,
|
||||
"builtin-only theme reaches row-0 `console` byte {col}"
|
||||
);
|
||||
}
|
||||
for col in 11..18 {
|
||||
assert_ne!(
|
||||
before[COLS + col].style.fg,
|
||||
BUILTIN,
|
||||
"parameter `console` byte {col} is lexically local"
|
||||
);
|
||||
}
|
||||
for col in 2..9 {
|
||||
assert_ne!(
|
||||
before[2 * COLS + col].style.fg,
|
||||
BUILTIN,
|
||||
"reference to shadowing parameter at row 2, col {col} is not builtin"
|
||||
);
|
||||
}
|
||||
|
||||
state
|
||||
.lua_host
|
||||
.lua()
|
||||
.load(
|
||||
r"
|
||||
local buf = pmacs.window.buffer()
|
||||
buf:replace(20, 27, 'logger')
|
||||
pmacs.parse._dispatch(buf, 'javascript')
|
||||
",
|
||||
)
|
||||
.exec()
|
||||
.expect("rename shadowing parameter");
|
||||
pump_async(&mut state, |s| {
|
||||
current_tree_text(s).as_deref() == Some(edited)
|
||||
});
|
||||
assert_eq!(
|
||||
current_tree_text(&state).as_deref(),
|
||||
Some(edited),
|
||||
"the edited JavaScript parse settled"
|
||||
);
|
||||
|
||||
let after = render_active_window_to_grid(&mut state, 5, COLS as u32);
|
||||
for col in 2..9 {
|
||||
assert_eq!(
|
||||
after[2 * COLS + col].style.fg,
|
||||
BUILTIN,
|
||||
"fresh local facts restore builtin styling at row 2, col {col}"
|
||||
);
|
||||
assert!(
|
||||
after[2 * COLS + col].style.bold,
|
||||
"fresh builtin capture reaches the rendered cell at row 2, col {col}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Sanity: the bundled `default_dark` theme produces a non-empty
|
||||
/// capture map and resolves common captures to non-default styles.
|
||||
/// Catches accidental regressions to a literally empty theme that
|
||||
|
|
|
|||
|
|
@ -0,0 +1,87 @@
|
|||
//! Cross-frontend tab-stop acceptance for core/TUI rendering.
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use pmacs::buffer::{Buffer, BufferId};
|
||||
use pmacs::cell::{Cell, CellCoord, CellGrid, CellSize, Glyph, Style};
|
||||
use pmacs::overlay::{BufferStyleOverlay, BufferStyleSpan, SharedBufferStyleSpans};
|
||||
use pmacs::text_view::TextView;
|
||||
use pmacs::view::{DisplayCoord, View, Viewport};
|
||||
|
||||
fn viewport(rows: u32, cols: u32, buffer_end: u64) -> Viewport {
|
||||
Viewport {
|
||||
buffer_start: 0,
|
||||
buffer_end,
|
||||
cell_origin: CellCoord::new(0, 0),
|
||||
cell_size: CellSize::new(rows, cols),
|
||||
gutter_w: 0,
|
||||
}
|
||||
}
|
||||
|
||||
fn render_text(buf: &Buffer, rows: u32, cols: u32) -> Vec<Cell> {
|
||||
let mut cells = vec![Cell::default(); (rows * cols) as usize];
|
||||
let mut grid = CellGrid {
|
||||
cells: &mut cells,
|
||||
stride: cols,
|
||||
size: CellSize::new(rows, cols),
|
||||
};
|
||||
TextView::new(buf).render(buf, viewport(rows, cols, buf.len()), &mut grid);
|
||||
cells
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn plain_text_projects_tabs_without_changing_source_bytes() {
|
||||
let source = b"\tx\n1234567\ty\n12345678\tz";
|
||||
let buf = Buffer::from_bytes(BufferId::next(), "tabs", source);
|
||||
let cells = render_text(&buf, 3, 20);
|
||||
let view = TextView::new(&buf);
|
||||
assert_eq!(view.pos_to_display(&buf, 1), Some(DisplayCoord::new(0, 8)));
|
||||
assert_eq!(view.pos_to_display(&buf, 11), Some(DisplayCoord::new(1, 8)));
|
||||
assert_eq!(
|
||||
view.pos_to_display(&buf, 22),
|
||||
Some(DisplayCoord::new(2, 16))
|
||||
);
|
||||
|
||||
for cell in cells.iter().take(8) {
|
||||
assert_eq!(cell.glyph, Glyph::Char(' '));
|
||||
}
|
||||
assert_eq!(cells[8].glyph, Glyph::Char('x'));
|
||||
assert_eq!(cells[20 + 8].glyph, Glyph::Char('y'));
|
||||
assert_eq!(cells[40 + 16].glyph, Glyph::Char('z'));
|
||||
|
||||
let mut retained = vec![0; buf.len() as usize];
|
||||
buf.snapshot_rope().slice(0, buf.len(), &mut retained);
|
||||
assert_eq!(retained, source, "rendering must not replace source tabs");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn buffer_style_overlay_covers_the_same_expanded_tab_columns_as_plain_text() {
|
||||
let source = b"a\tb";
|
||||
let buf = Buffer::from_bytes(BufferId::next(), "styled-tab", source);
|
||||
let mut cells = render_text(&buf, 1, 12);
|
||||
let spans: SharedBufferStyleSpans = Arc::new(Mutex::new(vec![BufferStyleSpan {
|
||||
start: 1,
|
||||
end: 2,
|
||||
style: Style {
|
||||
bold: true,
|
||||
..Style::default()
|
||||
},
|
||||
}]));
|
||||
let mut overlay = BufferStyleOverlay::new(spans);
|
||||
let mut grid = CellGrid {
|
||||
cells: &mut cells,
|
||||
stride: 12,
|
||||
size: CellSize::new(1, 12),
|
||||
};
|
||||
overlay.render(&buf, viewport(1, 12, buf.len()), &mut grid);
|
||||
|
||||
assert_eq!(grid.get(CellCoord::new(0, 0)).glyph, Glyph::Char('a'));
|
||||
assert!(!grid.get(CellCoord::new(0, 0)).style.bold);
|
||||
for col in 1..8 {
|
||||
let cell = grid.get(CellCoord::new(0, col));
|
||||
assert_eq!(cell.glyph, Glyph::Char(' '), "expanded tab column {col}");
|
||||
assert!(cell.style.bold, "overlay missed expanded tab column {col}");
|
||||
}
|
||||
assert_eq!(grid.get(CellCoord::new(0, 8)).glyph, Glyph::Char('b'));
|
||||
assert!(!grid.get(CellCoord::new(0, 8)).style.bold);
|
||||
}
|
||||
Loading…
Reference in New Issue