`COHERENCE.md` §9 grades the worker model "mechanism without identity",
and §0 names step 11 (background-work ownership) as one of the two
remaining thin ends of the golden journey. The mechanism half is solid —
cancellation, supersession, streaming, frame-aware draining, `*workers*`.
The identity half was absent: `PendingJob` carried no description of what
it was doing, `pmacs.workers.dispatch` discarded the registered handler
name three layers above anything that takes one, and §9's "no progress
indicator exists anywhere" was checkable and true.
Framing: `docs/worker-identity-framing.md` (revision 4, approved).
What lands:
**A required `purpose`, on the job and on the process.** Non-optional,
with no `Default`, so the compiler — not a test — is what proves every
dispatcher supplied one. `allocate` / `allocate_with_resource` collapse
into ONE private `JobSpec`-taking funnel (Q#W-1): the two-function split
existed only because one prior lane needed one extra parameter, and a
second lane doing the same produces `allocate_with_resource_and_identity`.
`register_external` gains a `purpose` parameter rather than deriving one,
because its `JobKind` is `McpRequest`/`LspRequest` for every method — a
category, not a description.
**A dispatch-name ambient (Q#W-2), read at that same single funnel.** The
capture point is Rust, not the Lua wrapper layer, because a handler
reaching straight for `pmacs._async._dispatch_*` bypasses the wrappers
entirely — and those are precisely the callers attribution exists for.
Seven rules; the ones that decide whether it is honest:
- **Rule 1 — the extent is NON-YIELDABLE, and that is ENFORCED.** Both
supported yield APIs refuse inside it, modelled on the `commit_to`
refusal already in `async.lua`. The guards reject BEFORE parking and
reject UNCONDITIONALLY: one placed after `_is_complete` would fire only
when a yield really occurred, passing under test and failing
intermittently in production.
- **A raw `coroutine.yield` is NOT covered, and nothing here claims it
is.** R46 is a convention, and the scheduler inspects the yielded value
only after `coroutine.resume` returns — by which point the coroutine has
already suspended — so no refusal sited in a yield helper is ever
consulted. The residual is recorded in the framing §2 and in the
suite's module docs rather than papered over with a test that would
imply coverage this design lacks.
- **Rule 5 — unwind-safe.** A raising handler still pops. A version that
did not would let one failure poison every later dispatch in the session
with a stale name: the feature would stop failing loudly and start lying
silently. The bracketing also has to preserve the tail call it replaced:
`dispatch` was `return handler(args, opts)` and propagated EVERY return
value, so the pop/rethrow runs behind a varargs boundary rather than a
`local ok, result = pcall(...)` that would silently truncate a
multi-value handler. Varargs rather than `table.pack`, because that is
Lua 5.2 surface and LuaJIT is this project's default backend.
- **Rule 6 — compose, do not replace.** `"<name>: <purpose>"`, because
letting the dispatcher's purpose win loses the third party again and
letting the name win discards the only description of the actual work.
**A statusline activity indicator** — the fourth `pmacs.statusline.register`
adopter, after `mode`, `terminal` and `lsp`. A count plus the OLDEST
in-flight job's purpose ("busiest" is not a defined quantity; jobs carry
no cost estimate), and **absent entirely** when idle rather than a
zero-width segment that costs modeline width forever to say nothing is
happening. Gated by one setting, `ui.activity-indicator` (boolean, default
true, Q#W-6) — a permanently-visible modeline element is a preference
someone genuinely holds on day one. No setting for purpose capture
itself: that is substrate.
**NO WIRE CHANGE.** The indicator rides the existing `StatuslineSegments`
vector, so a fourth provider adds an element, not a variant.
`PROTOCOL_VERSION` and `ADVERTISED_PROTOCOL_VERSION` are untouched — which
is the property that lets this run beside the two lanes holding the bump
slot.
**Q#W-7 — a pre-existing defect, repaired here, and NOT one anybody has
observed.** `Handle:await()` refuses inside `pmacs.window.commit_to`
precisely so a coroutine cannot park with the frontend scope pushed
(Journey Stage 1a, Q#JR14b). But `pmacs.async.yield_to_next_tick()` also
yields, is public, and carried no such refusal — so that invariant had a
second entrance, and a coroutine could produce exactly the misrouting the
`await` guard exists to prevent. It gains both refusals here: the same
supported yield helper, the same invariant, the same edit family, so
splitting it would have preserved a known hole without reducing
integration risk.
**Reachability by a real caller is UNPROVEN.** This was found by reading
the guard family while scouting rule 1, not by reproducing a fault. No
production caller is known to yield through that door inside a commit,
and the test pins the guard rather than reproducing a user-visible bug.
Nobody should later cite this commit as evidence the bug was observed in
the wild. Its witness is a PAIR, like rule 1's: the refusal fires **and**
the commit scope is restored afterwards — a guard that raises while
leaving the scope pushed converts a silent fault into a loud one and
fixes neither.
`journey_acceptance` carries the established `commit_to` pins —
forged-destination refusal, scope-and-restore on normal return and on
raise, the await refusal, delivery to the requesting frontend. It passes
**untouched**, which is what says this closed a gap in Journey Stage 1a's
semantics rather than altering them.
What is deliberately NOT here, and why it is worth saying:
- **No `owner`, in any spelling** — not `origin`, not `subsystem` (§3).
Populated from static per-subsystem constants it would be an origin,
not an owner, and would confidently misattribute third-party work to a
builtin at exactly the point §9 wants attribution. A field that asserts
a falsehood is worse than an absent one. The slot stays empty until P3
can fill it with a real package signal.
- **No `parent`** (Q#W-5). An unpopulated field renders as `None`
everywhere and reads as "this job has no parent" rather than "this
system does not track parents". Stage 3 builds the lifetime model and
the field together.
Consequences worth recording:
- `ProcessSpec::new` takes a third argument. The 40-odd call sites are
almost all tests; the three production ones (LSP, MCP, terminal) supply
real descriptions. `pmacs.process.spawn`'s Lua surface keeps `purpose`
OPTIONAL, falling back to the label — requiring it there would break
every existing caller for no coverage the compiler is not already
providing, and a caller's own label is not a fabrication.
- `pmacs.process.list` gains a `purpose` KEY on each row and enumerates
exactly the same processes (Q#W-4). Terminal PTYs stay hidden: three
acceptance suites use `#pmacs.process.list()` as a leak baseline, and
widening the accessor would inflate all three. Stage 2's unified view
owns that decision.
- `statusline_segments_acceptance`'s builtin-provider inventory grows to
`["activity", "mode", "terminal", "lsp"]`. That assertion exists to
grow when a builtin provider is added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bqGA6s9tTUFzYpbeW3tai
All four findings were the same shape: a failure that left state wrong
and told nobody.
**Delete refusals reach the user.** `reconcile_delete_and_fire` returned
`kept_modified` and `refused` and both production callers discarded
them, so a last-buffer refusal or the asynchronous modified-buffer race
left the file gone and the buffer still bound to it — and the next
`C-x C-s` recreates the deleted file. Reporting now happens inside the
shared seam, not at its call sites, for the same reason the
reconciliation does: a caller that has to remember to report is a caller
that will forget. The message names the buffers (capped, with a count
for the rest) and states the consequence, and it is written to
`EditorCore::status`, not `pmacs.error` — that channel is defined only
by a test stub, so a report there would be the same silence.
`reconcile_delete` now prefixes `kill_buffer`'s reason with the buffer
name, because "cannot kill the last remaining buffer" does not say which
buffer is now bound to a deleted path.
**The LSP subscribers stop swallowing their own failures.** Ignored
`pcall`s around `did_close`, `forget_uri`, `did_open` and overlay
re-rooting made the callback return successfully, so the
`all-must-succeed` logger had nothing to log — concretely, a stale server
made `forget_uri` raise while the callback carried on with the old
stores, routes and `documents` entry all live. A shared failure sink
attributes each step, reports on both channels, and raises **after** the
loop, so one unreachable server cannot leave every other attachment
unreconciled.
**`forget_uri` abandons requests through the established path.** It
purged `pending_routes` and `pending_external` but not the same ids
`send_request` put in `LspClient.pending`, and recorded nothing in
`cancelled_rids`. The per-rid work is extracted from
`drain_cancelled_externals` as `abandon_request` and reused, rather than
a second incomplete copy: route, client pending, cancelled record and
`$/cancelRequest` now happen together.
**Acceptance 35 is pinned.** With a plain delete the forbidden fallback
was unobservable — `find_or_open` raises out of `load_file` and the
`pcall` swallows it — so both assertions passed with the fallback
present. The plan now deletes the origin's file and recreates it, which
gives the fallback something to open and makes "restores nothing"
falsifiable. The corrected G1 explanation also reaches the production
comments, which still repeated the false `resolve_target_buffer::NotFound`
story.
New pins: acceptance 53 and 53b assert the status channel; a stale-server
row asserts attribution on both channels *and* that the healthy
attachment still reconciles; an `lsp.rs` unit test asserts the client-side
abandonment with an unrelated request as its control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lv428Fth9LRtffwJSsqH7T
`tests/resource_reconciliation_acceptance.rs`, 23 rows, no dired
content — items 23–37 and 50–55 driven through the real entry points:
`pmacs.fs.rename` / `pmacs.fs.remove` fire-and-forget for the drain
harvest, `pmacs.buffer.apply_resource_op` for the synchronous arm, and
the fake server's `workspace/applyEdit` for the applier.
The rows that took design rather than transcription:
Item 27 opens two descendants AND two buffers on one exact path, since
one child would not defeat a first-match lookup. Item 29 tests name
provenance in both directions, including a name explicitly set to a
string that normalizes to the file's own path — the case a
path-equivalence heuristic gets wrong. Item 30 paints a real frame and
counts diagnostic underlines per window rect, because
`DiagnosticView.uri` is private and a store assertion would prove
nothing about re-rooting; it also pins each overlay's index in the
composition order, which is what a remove-and-re-push breaks. Item 53b
states its three assertions individually, since a compound check can
pass on two of the three.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lv428Fth9LRtffwJSsqH7T
`src/lsp.rs` gains nine: the fourteen-family store inventory with a
17-entry precondition so it cannot pass vacuously, the route purge with
`workspace/symbol` and another server's route both surviving, the
awaiter drain joined on the rid, the error contract's two arms, the
late-publish drop with its does-not-over-reach companion, the
`mark_document_stale` gate across all three stale stores, exact-pair
tombstone identity, and reclamation under both `start_generation` and
terminal `forget`.
`src/diag.rs` pins that `forget` drops the epoch while `clear`
deliberately bumps it — the leak a `clear`-based forget would leave in
the one map nothing prunes.
`src/async_runtime.rs` injects two resource replies onto the private bus
in each order and asserts `TickOutcome.resources` reports arrival order,
not allocation order; plus that a failed or cancelled mutation is not
harvested at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lv428Fth9LRtffwJSsqH7T
One shared walk query (`buffers_bound_under`), lifted out of #190's
`delete_verdict` so the guard and both reconciliation seams cannot
disagree about which buffers an operation touches: every buffer, both
sides normalized, component-aware containment.
`EditorCore::reconcile_rename` moves the stored path and — only for a
`PathDerived` name — the buffer name. `EditorCore::reconcile_delete`
composes the same two removal phases `pmacs.buffer.kill` composes,
preflighting `editing_in_progress` because a `ConcurrentEdit` refusal
arrives after `kill_buffer` has already moved windows. Phase 2 stays
with the caller; `EditorCore` gains no Lua handle.
`AsyncRuntime::tick` now returns a `TickOutcome` carrying the settled
ids plus the successful resource mutations, in bus-arrival order, which
is documented as not being execution order. `PendingJob.resource`
retains the paths the dispatchers move into the worker closure.
`LspManager::forget_uri` purges the routes carrying a URI, drains the
awaiters joined to them on the rid, and clears all fourteen stores plus
`documents`. A generation-scoped exact-pair tombstone gates the two
uncorrelated writers that can otherwise resurrect what it cleared:
`publishDiagnostics` and `mark_document_stale`, which now takes a
server id. `ResponseRoute::scoped_uri` is the one variant list, with
`uri()` delegating to it.
New Lua surface: `pmacs.buffer.set_name`, `pmacs.lsp.forget_uri`,
`pmacs.diag._rename_resource`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Lv428Fth9LRtffwJSsqH7T
Add the strict pmacs.statusline provider registry, deterministic
borrow-released per-window evaluation, context-scoped failure latches,
and a pure built-in LSP provider.
Preserve the legacy TUI modeline while composing faced custom runs,
and append authoritative complete StatuslineSegments replacements for
semantic frontends. Expand dynamic ThemeFacts, reset producer/frontend
baselines symmetrically, and gate all provider work off protocol v18.
Teach the GPU to atomically validate, resolve, shape, clip, and cache
custom modeline runs without displacing the protected status suffix.
Document the public Lua lifecycle, wire ownership, snapshot semantics,
and the fully gated Arc 4 stage-3 delivery state.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Preserve PR #123's unpushed review fixes on a transfer branch: initial
didChangeConfiguration delivery, explicit JSON validation, the pinned
JSON server provider, corrected YAML configuration sections, and
deterministic plus real-provider acceptance coverage. Record the
observed yaml-language-server 1.24.0 standalone smoke and leave the
real YAML-through-pmacs test, rebase, and full gates explicitly pending
for the destination machine.
request_rename and request_prepare_rename sent raw byte columns instead
of routing through outbound_position — the same bug class as the
semantic-range and code-action fixes that just merged (#105). On a
UTF-16 server, a rename at a position past non-ASCII text resolves the
wrong character (or an invalid one) and renames the wrong symbol.
Both single-Position builders now convert. The posecho fake validates
request positions on its rename/prepareRename arms in UTF-16 units, and
the new test drives both requests at byte offset 3 of "éx" (UTF-16
character 2) — both stores filling proves both builders converted.
(Fix authored locally by Levi during the round-5 review; recovered from
the working tree after the #105 merge and landed verbatim, plus a
cargo fmt pass.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses the round-5 finding: the whole-document range that serves a
RANGE-ONLY semantic-token provider derived its columns from UTF-8 byte
counts and sent them unchanged — unlike the inlay path, it skipped
outbound_position. A UTF-16 server receives an invalid end character
for non-ASCII text ("é" is two bytes, one UTF-16 unit) and may reject
the request; since /range is a range-only provider's ONLY pull path,
that means no semantic styling at all.
Both bounds of request_semantic_tokens_range now go through
outbound_position. request_code_action had the identical bug (byte
columns, no conversion) and is fixed in the same stroke — same class,
same one-line shape, commented as such.
Fixture: `rangeonly16` fake mode = rangeonly + negotiated UTF-16 +
STRICT UTF-16 bounds validation on /range (fail-closed: a missing
didOpen record or absent uri also rejects, so the fixture can never
pass vacuously). An env-gated PMACS_FAKE_RANGE_SINK records the
received range for debugging. Test opens a file whose last line ends
in non-ASCII and asserts tokens arrive; verified it bites — with the
conversion removed the wire carries the byte column (13 vs the valid
11), the fake rejects, and the test fails.
Honest note: an earlier bite-check in this session produced a vacuous
pass because short, non-unique edit patterns hit the WRONG json! block
(temporarily regressing the inlay conversion and accidentally
converting code-action). The final diff is anchored uniquely and
verified: inlay unchanged (whitespace only), semantic + code-action
converted, bite-check red/green confirmed against the exact lines.
Gates: fmt + workspace clippy clean; lib 1500; crdt 1672; m4 99;
killring 30; completion 9; GPU 58; git diff --check clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Full-document didChange went out per keystroke: three O(file) copies,
O(file) JSON, and a BLOCKING pipe write on the daemon main thread
(Linux pipe buffers are 64KiB; a 240KB notification stalls the frame
loop until the langserver drains). The dominant daemon-side typing
cost on large files, and freeze-class when a server stops reading.
- lsp.lua: the after-edit hook now bumps the version, marks the
cached render families stale (new _mark_document_stale binding, so
stale suppression stays keystroke-accurate), and records the buffer
dirty. The coalesced send fires on the async tick after 75ms of
quiet, or at most 400ms behind during continuous typing. Anything
that consults the server flushes first (attached_for_active,
repull_for_attachments, pull_inlay_hints_quiet) so requests and
position-encoding conversion never see stale text. Versions may
skip values; LSP only requires they increase.
- Inlay hints re-pull at flush cadence: they're pull-model, nothing
re-requested them after edits, so hints died on the first
keystroke and never returned.
- process.rs StdinWriter: a per-generation writer thread owns the
child's stdin; write_stdin queues and never blocks (64MiB budget
converts a wedged child into an error); close_stdin drains then
EOFs, preserving the MCP flush-then-EOF contract.
- pmacs.editor.monotonic_ms + pmacs.lsp._flush_did_changes bindings;
acceptance test pins burst-coalescing, flush-on-demand, and the
quiet-window tick flush.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Surfaced during session-5 manual validation as the final iteration
of bet #1 from the framing pass: edits that shift byte positions
left old diagnostic colors painted over post-edit text in both
`pmacs-gpu` and (now-visible) the TUI. Persisted for the full LSP
re-analysis window (100ms–5s).
PR #47 fixed the StyleSpans side via generation-tracked full=true
emission, but Decorations remained vulnerable: the producer's diff
shipped old diagnostics from the diag store, whose entries were
indexed at pre-edit byte positions until clangd republished.
Fix: a per-URI `stale_uris` flag in `DiagnosticStore`. The LSP
layer's `did_change_full` marks the URI stale right after sending
the notification; the next `publishDiagnostics` absorb path's
`set` clears it. The `semantic_render` producer reads `is_stale`
and skips diagnostic emission entirely while stale.
Effect: between an edit and clangd's next publish, the producer
ships zero diagnostic decorations. Frontend's replace/merge clears
old positions cleanly. Brief uncolored window (≤ LSP re-analysis
latency) replaces the previous wrong-position-color persistence.
The correct visual tradeoff: honest emptiness over deceptive
staleness.
Files changed:
- `src/diag.rs` — `DiagnosticStore` gains `stale_uris: HashSet<String>`;
new `mark_stale` / `is_stale` API; `set` and `clear` reset the
flag on the assumption that absorption / explicit removal mean
the LSP has caught up.
- `src/lsp.rs` — `LspManager::did_change_full` calls
`diag_store.lock().mark_stale(uri)` after `send_notification`.
- `src/semantic_render.rs` — `scoped_decorations` reads `is_stale`
alongside `for_uri`; when stale, suppresses the diagnostic
loop (selection and other non-diagnostic kinds still emit).
Tests (all crdt-gated where they reference semantic_render):
- `diag::tests::stale_flag_default_false`
- `diag::tests::mark_stale_sets_flag` (per-URI scoping)
- `diag::tests::set_clears_stale_flag`
- `diag::tests::empty_set_clears_stale_flag_too`
- `diag::tests::clear_drops_stale_flag`
- `semantic_render::tests::diagnostics_suppressed_while_diag_store_stale`
— assert no diagnostic kinds emit while stale; assert they
re-emit after a fresh `set` clears the flag.
Gates: cargo fmt + clippy (workspace, with/without `crdt`) clean;
lib 1482 (+6) with crdt; 1319 (+6) without; m4 83; m11_5 2.
This is approach (A) from the session-5 ask: track per-URI freshness
relative to buffer edits, suppress emission until LSP catches up.
Approach (B) — clangd's didChange/publishDiagnostics version
matching — would be more precise but requires plumbing version
tracking through the LspManager's document state, which is a
larger change deferred. The stale-flag captures the same semantic
("any edit since last publish ⇒ stale") at a cheaper cost.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Languages with no bundled tree-sitter grammar (C/C++, …) had empty
StyleSpans — the visible "no C++ syntax coloring" gap. scoped_style_spans
now applies per-language styling authority (policy A): a grammar-backed
language stays tree-sitter-only (unchanged); a grammar-less buffer falls
through to lsp_scoped_style_spans, which reads the semantic-token store
(for_uri), resolves the owning server's encoding + legend via
LspManager::semantic_style_context, converts UTF-16 start/length to byte
per line with char_to_byte (now pub(crate); semantic-token data is NOT
byte-rewritten upstream, unlike inlay hints — see Step 0), names the
token via the legend, maps through the existing Theme::lookup, and drops
default-style spans. Output is shape-identical to the tree-sitter path,
so the M11.4 diff pipeline consumes it unchanged. Never two authorities
on one buffer: a still-parsing grammar-backed buffer returns empty
rather than briefly borrowing LSP styling.
Step 4 folded in: golden tests in the semantic_render module —
cpp_style_comes_from_lsp_when_no_tree_sitter_grammar (headline),
suppression (M11.4 reuse), incremental-on-token-change, and honest
empty-without-tokens. A #[cfg(test)] LspManager::insert_initialized_
test_client supplies a synthetic Initialized client (legend caps +
encoding) with no process, so the producer path is exercised without
a live server.
Per-byte tree-sitter/LSP blend and multi-server-same-uri merge remain
deferred open questions (recorded in the design note by Step 5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dynamic file-watch registration with a full snapshot-diff watcher.
- src/lsp.rs: did_change_watched_files(sid, &changes) notification;
capability workspace.didChangeWatchedFiles.dynamicRegistration=true
(mandatory — clangd/rust-analyzer/gopls only register dynamically).
- src/lua_bindings.rs: pmacs.lsp.did_change_watched_files binding.
- builtin/runtime/lsp.lua: client/(un)registerCapability handled in
the server-request pump (reply null; start/stop watchers). Brace-
expanding glob → anchored Lua pattern; recursive read_dir/stat
snapshot-diff poller emitting per-file created/changed/deleted
filtered by glob + WatchKind, batched into one notification;
self-cancels when the server dies or unregisters. luajit-safe
(kind_has() arithmetic, no 5.3 bitwise).
- pmacs_fake_lsp.rs: `filewatch` mode registers a **/*.txt watcher
and logs received changes to <base>/.received (disk side-channel —
the protocol stream is drained by the pump).
- tests/m4_acceptance.rs: m4_24 asserts create(1)/change(2)/
delete(3) for matching .txt only; non-matching .md filtered.
Bug caught in validation: `**/` → `(.*/)?` is not a valid Lua
pattern (no group quantifier) — matched nothing, zero events. Fixed
to `**/`→`.-`, `**`→`.*`; m4_24 surfaced it.
client/unregisterCapability cancels watcher records (code-reviewed);
not asserted in m4_24 — a "no further notifications" negative-timing
check is flaky; the create/change/delete + filter path is the
deterministic proof.
Gates: lib 1301/0, m4 79/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m5_8 5/0, m11_5 (--features crdt) 2/0; fmt + clippy
clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
send_notification wrote frames straight to stdin regardless of
lifecycle state. At CLI startup the buffer.after-load hook fires
did_open while clangd's initialize is still in flight; the LSP
spec lets a server discard any notification before the
initialize/initialized handshake, and clangd does. The document
is then never "added", so every later request fails with
`-32602 trying to get AST for non-added document` (and no
diagnostics ever appear). Lenient servers (rust-analyzer, gopls)
queue internally, which is why the M4.5 arc didn't catch it.
Buffer notifications issued while Starting/Initializing on the
client and replay them, in issue order, immediately after the
`initialized` notification goes out (flush_deferred_notifications,
from the initialize-response handler). `initialized`/`exit` are
sent directly by the lifecycle handler and never pass through
send_notification, so they bypass the gate. The queue is cleared
on (re)start_generation since the reattach path re-sends fresh
did_opens against the new process.
This also makes the stale lsp.lua:211 comment ("the manager
queues it cleanly even while starting/initializing") finally true.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backlog item 2 — perf refinement over the v1 full-only request.
- src/semantic_tokens.rs: SemanticTokensResponse retains the raw
int stream; factored decode(); new apply_delta(prev_raw, v)
splices a SemanticTokensDelta (edits:[{start,deleteCount,data}])
over the previous raw — descending-start application so unordered
server edits stay valid, bounds clamped, spec-allowed
full-instead-of-delta detected and parsed. +4 unit tests.
- src/lsp.rs: request_semantic_tokens_range (reuses the
SemanticTokens route/store) and request_semantic_tokens_delta
(new ResponseRoute::SemanticTokensDelta; absorb splices against
the store's retained raw). Capability upgraded to
requests:{ full:{ delta:true }, range:true }.
- src/lua_bindings.rs: _request_semantic_tokens_range_raw,
_request_semantic_tokens_delta_raw,
pmacs.semantic_tokens.result_id(sid,uri).
- builtin/runtime/lsp.lua: range/delta wrappers;
pmacs.lsp.semantic_tokens() auto-prefers delta when a prior
result id exists (else full), no longer clears the store (delta
needs the retained raw), tags the modeline "(delta)". The range
wrapper is exposed without a default command (no viewport source
in the bundle yet).
- pmacs_fake_lsp.rs: /range and /full/delta arms (delta is an
edit script over the /full data).
- tests/m4_acceptance.rs: m4_20 (range decode), m4_21 (full seeds
rid-1; delta against it splices to the updated 3rd token + rid-2).
Gates: lib 1289/0, m4 76/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m5_8 5/0, m11_5 (--features crdt) 2/0; fmt + clippy
clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backlog item 1, combined (1a+1b) now that semantic tokens (#23) is
on main. Lets servers tell us cached inlay hints / semantic tokens
are stale and have the client re-pull, instead of the on-demand-
only v1 model.
- src/lsp.rs: advertise workspace.inlayHint.refreshSupport=true and
workspace.semanticTokens.refreshSupport=true.
- builtin/runtime/lsp.lua: generalize the L3 workspace/applyEdit
pump into handle_server_requests; add branches for
workspace/inlayHint/refresh and workspace/semanticTokens/refresh
— reply null per spec, then repull_for_attachments re-issues the
matching request (request_inlay_hint / request_semantic_tokens)
for every attached document on that server. Fire-and-forget; the
response absorbs via its existing route like the command path.
Only attachment servers are drained (directly-spawned test
servers untouched).
- pmacs_fake_lsp.rs: `inlayrefresh` / `semantictokensrefresh`
modes send the respective server→client refresh request at
`initialized` (mirrors the wsconfig pattern).
- tests/m4_acceptance.rs: m4_18 / m4_19 attach via config and
assert the store populates purely from the server-driven refresh
chain — no explicit inlay_hints()/semantic_tokens() call.
Gates: lib 1285/0, m4 74/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m5_8 5/0, m11_5 (--features crdt) 2/0; fmt + clippy
clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
LSP data layer only — independent of the M11 semantic-render
protocol (semantic_render.rs / semantic_client.rs, tree-sitter →
frontend wire families). No InstanceMessage family added; wiring
LSP tokens into styling is a separate rendering milestone. Same
shape as every sibling LSP feature: typed store + async request
+ Lua surface + command + modeline summary.
- src/semantic_tokens.rs: decode the 5-int relative encoding
(deltaLine, deltaStartChar, length, tokenType, tokenModifiers)
into absolute SemanticToken{line,start,length,token_type,
token_modifiers}, with the same-line-vs-new-line deltaStartChar
rule and defensive truncation of a malformed trailing group.
SemanticTokensLegend::from_capabilities parses
semanticTokensProvider.legend and resolves type index / modifier
bitset to names. Store keyed (server, uri). 6 unit tests.
- src/lsp.rs: store + accessor, ResponseRoute::SemanticTokens +
absorb, request_semantic_tokens (/full; v1 no range/delta),
textDocument.semanticTokens client capability (full-only,
formats=[relative], standard LSP legend).
- src/lua_bindings.rs: _request_semantic_tokens_raw,
pmacs.semantic_tokens.{tokens, legend, clear} (legend reads the
per-server initialize capabilities).
- pmacs_fake_lsp.rs: semanticTokensProvider.legend in initialize;
textDocument/semanticTokens/full arm with relative-encoded data.
- builtin/runtime/lsp.lua: pmacs.lsp.semantic_tokens() requests
full, stores, modeline summary (first token's type resolved via
legend); lsp.semantic-tokens command + C-c y.
- tests/m4_acceptance.rs: m4_17 drives the request via the Lua
surface, asserts decoded absolute tokens (incl. deltaLine!=0 ⇒
absolute startChar) and legend index→name resolution.
Gates: lib 1285/0, m4 72/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m5_8 5/0, m11_5 (--features crdt) 2/0 (confirms no
collision with the M11 render protocol); fmt + clippy clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Independent LSP feature (not part of the L1-L4 cross-file arc),
shipped in the same shape as every sibling: typed store + async
request + Lua surface + command + modeline summary, with the
inline renderer deferred as its own milestone.
- src/inlay_hint.rs: parse InlayHint[]|null — position, label
(string OR InlayHintLabelPart[] flattened), kind (type/
parameter), paddingLeft/Right, tooltip (string|MarkupContent).
Store keyed (server, uri). 5 unit tests.
- src/lsp.rs: inlay_hint_store + accessor, ResponseRoute::InlayHint
+ absorb, request_inlay_hint (range params), textDocument.
inlayHint client capability (no resolveSupport/refreshSupport —
full hints, on-demand re-query is the v1 model).
- src/lua_bindings.rs: _request_inlay_hint_raw,
pmacs.inlay_hint.{hints,clear}.
- pmacs_fake_lsp.rs: textDocument/inlayHint arm returning a
string-label type hint and a label-parts parameter hint.
- builtin/runtime/lsp.lua: pmacs.lsp.inlay_hints() requests over
the whole-buffer range, stores, modeline summary;
lsp.inlay-hints command + C-c i; scope header notes the inline
renderer is a later milestone.
- tests/m4_acceptance.rs: m4_16 drives the request via the Lua
surface, asserts both label shapes / kinds / padding parsed.
Deferred (scoping, not a regression): inline virtual-text
rendering. The VirtualCellOverlay model only overwrites existing
cells; rendering hints inline needs a column-inserting/reflowing
renderer — a rendering milestone, not an LSP task — staged like
the hover panel / references list. pmacs.inlay_hint is the data
surface a future render layer subscribes to.
Gates: lib 1279/0, m4 71/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m5_8 5/0, m11_5 (--features crdt) 2/0; fmt + clippy
clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Third cross-file layer: code actions and the server→client edit
channel that executeCommand-driven actions depend on.
- src/code_action.rs: normalise (Command | CodeAction)[] into one
CodeActionItem (bare-Command vs nested-Command disambiguated by
whether top-level `command` is a string; inline `edit` reuses
rename::WorkspaceEditResponse). Store keyed (server, uri). 5 tests.
- src/lsp.rs: code_action_store + accessor, ResponseRoute::CodeAction
+ absorb, request_code_action, request_execute_command (awaiter
only — effect arrives out of band). Capabilities: codeAction
(+codeActionLiteralSupport), workspace.executeCommand, and
workspace.applyEdit flipped to true.
- src/lua_bindings.rs: _request_code_action_raw,
_request_execute_command_raw, _parse_workspace_edit (any raw
WorkspaceEdit JSON -> applier input shape), pmacs.code_action.*.
- pmacs_fake_lsp.rs: textDocument/codeAction arm (command action
first, inline-edit action second) + workspace/executeCommand arm
that emits a server→client workspace/applyEdit before responding.
- builtin/runtime/lsp.lua: pmacs.lsp.code_actions (apply first
action: inline edit and/or executeCommand); the applyEdit pump
(chained on pmacs._async.tick, drains only attachment-server
events, snapshots server ids before applying since find_or_open
can mutate `attachments`, replies { applied }); lsp.code-actions
command + C-c a keybind.
- tests/m4_acceptance.rs: m4_14 drives the full
codeAction→executeCommand→applyEdit chain end to end.
Gates: lib 1273/0, m4 69/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m11_5 (--features crdt) 2/0; fmt + clippy clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Second cross-file layer on the L1 foundations: a multi-file
WorkspaceEdit applier and an LSP rename UX driving it.
- src/rename.rs: parse a WorkspaceEdit (both `changes` map and
`documentChanges`, the latter preferred per spec; AnnotatedTextEdit
handled; create/rename/delete resource ops counted into
`unsupported_ops` for L4) into per-file TextEdit lists. RenameStore
keyed by the request's origin URI. 6 unit tests.
- src/lsp.rs: rename_store + accessor, ResponseRoute::Rename,
request_rename, and the textDocument.rename client capability
(prepareSupport=false — L2 renames from the cursor position).
- src/lua_bindings.rs: _request_rename_raw + pmacs.rename.{file_edits,
unsupported,clear}.
- pmacs_fake_lsp.rs: textDocument/rename arm; `rename` mode returns a
2-file documentChanges plus a create resource op.
- builtin/runtime/lsp.lua: apply_workspace_edit (preflight rejects
unresolvable URIs before mutating anything; per-file reverse-sorted
application; origin buffer restored), pmacs.lsp.rename with a
minibuffer prompt, lsp.rename command, C-c r keybind.
- tests/m4_acceptance.rs: m4_13 drives rename end-to-end through the
minibuffer and asserts both files mutated + origin restored.
Gates: lib 1268/0, m4 68/0, m8_1 10/0, m8_9 26/0, m8_10 19/0,
m9_1 18/0, m11_5 (--features crdt) 2/0; fmt + clippy clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Completes the read-only LSP feature set (everything except edits).
Same request→store→Lua pattern as the nav batch; these three need
new parsing (new response shapes), so they were deliberately split
from the Location-family PR (#16).
- src/symbol.rs: one flat Symbol type for both symbol requests.
from_lsp_value handles BOTH LSP shapes — hierarchical
DocumentSymbol[] (flattened with depth + parent chain) and flat
SymbolInformation[]/WorkspaceSymbol[] (location.uri, range
optional for WorkspaceSymbol). Scope-keyed (Document(uri) vs
Workspace(query)) so an outline and a query don't collide.
- src/document_highlight.rs: range + DocumentHighlightKind (absent
defaults to Text=1 per spec), (server,uri)-keyed.
- lsp.rs: three ResponseRoute variants + absorb arms + request
methods. documentSymbol/documentHighlight ranges convert via the
requested-doc codec; workspace/symbol results are cross-file →
route uri "" → non-destructive passthrough (same rule as
cross-file definition).
- Lua: raw bindings + pmacs.document_symbol / .workspace_symbol /
.document_highlight read surfaces (the new LSP Symbol is aliased
to avoid the pre-existing project_index::Symbol name clash);
lsp.lua wrappers + an lsp.document-symbols command on C-c o
(modeline summary; outline buffer is future UX).
- Tests: 6 parser unit tests (hierarchical depth/parent, flat
SymbolInformation, range-less WorkspaceSymbol, highlight kind
default, scope non-collision) + an e2e driving all three through
the async bridge asserting shape correctness.
Also includes a pre-existing rustfmt normalization of the #15
semantic-frontend files (protocol.rs / semantic_client.rs /
semantic_render.rs) — main was not rustfmt-clean there after the #15
merge; bundled here per operator decision so the fmt gate is green.
Gate: fmt clean; clippy --all-targets -D warnings clean; lib
1257/0; m4_acceptance 66/0; m9_1 18/0; m8_1/m8_9/m8_10 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "cheap batch" subset that is genuine template-fill: these four
return the exact `Location | Location[] | LocationLink[] | null`
shape `textDocument/definition` already parses, so no new parsing —
only a kind discriminator so they don't collide on (server, uri).
- src/locations.rs: a (server, uri, kind)-keyed store whose value
type is the reused crate::definition::DefinitionResponse. The
proven definition store + Lua API are untouched.
- lsp.rs: ResponseRoute::Locations { uri, kind }; one absorb arm; a
DRY request_locations helper + request_references /
request_declaration / request_type_definition /
request_implementation. references sends
context.includeDeclaration. Supersede keys derive from each
kind's distinct method, so the four don't cancel each other.
- Lua: _request_*_raw bindings + install_locations exposing
pmacs.references / .declaration / .type_definition /
.implementation ({ locations, clear }, mirroring pmacs.definition,
reusing definition_response_to_lua). lsp.lua Handle wrappers + a
lsp.find-references command bound to M-? (modeline summary;
references-list buffer is future UX, like the hover panel).
- Tests: locations.rs unit tests (kind labels distinct; keys don't
collide); e2e driving all four through the async bridge and
asserting each routes to its own kind slot (fake returns distinct
lines 11/21/31/41).
Scoped: documentSymbol / workspaceSymbol / documentHighlight return
different shapes (new parsing) — a separate follow-up, not crammed
in here.
Gate: fmt clean; clippy --all-targets -D warnings clean; lib
1232/0; m4_acceptance 65/0; m9_1 18/0; m8_1/m8_9/m8_10 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
gopls / pyright / basedpyright / clangd issue server→client
`workspace/configuration` during startup and degrade (or fall back
to defaults) without a reply. pmacs advertised `configuration:false`,
so it never got the chance.
- Advertise `workspace.configuration: true`.
- New `settings` field on the spawn spec, threaded through
lua_to_lsp_spec → ensure_server (pmacs.lsp.config[lang].settings).
- handle_request intercepts `workspace/configuration` (mirrors the
publishDiagnostics interception in handle_notification): each
item's dotted `section` resolves against the server's settings via
resolve_config_section; one array element per item; unknown
sections answer `null` (the spec's "not configured" signal,
distinct from a configured null). All other server→client requests
still surface as a `Request` event for the consumer.
- The Python default now ships
`python.analysis.typeCheckingMode = "basic"` (+ basedpyright.*
alias), so the #12 basedpyright-noise concern is now actually
fixed rather than only documented; a project pyrightconfig.json /
[tool.pyright] still wins where present.
Scoped: `scopeUri` ignored (single-root; same settings regardless
of scope) until multi-root, a separate deferred item.
Tests: exhaustive resolve_config_section unit test (dotted paths,
configured-null vs unknown-null, whole-object for absent section);
new `wsconfig` fake mode pulls config at `initialized` and echoes
pmacs's answer back; end-to-end test asserts the configured section
round-trips.
Gate: fmt clean; clippy --all-targets -D warnings clean; lib
1230/0; m4_acceptance 62/0; m9_1 18/0; m8_1/m8_9/m8_10 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SemanticRenderState now also projects InstanceMessage::Decorations
from the instance-side state pmacs actually has.
- Selection: per-window byte-native state via active_window_for(fid)
(SemanticRenderState now carries the session FrontendId), gated to
the declared buffer and clipped to the viewport →
DecorationKind::Selection.
- Diagnostics: the shared DiagnosticStore keyed by file URI. Made
lsp::path_to_file_uri pub(crate) (byte-identical to the Lua
file_uri_for) so the projection reproduces the exact store key from
core.file_path. LSP (line,col) -> byte via a line-start scan
against the buffer source; severity -> DiagnosticError/Warning/
Info/Hint. Clipped to the viewport.
- StyleSpans and Decorations suppress unchanged frames independently
(separate last_* maps): a selection move doesn't force a styling
re-send and vice versa.
- Deliberately NOT emitted: SearchMatch/SearchMatchActive (no
instance search-hit store), CurrentLine (frontend derives from
CursorByte; emitting it would breach the contract boundary).
- InlineAdornments/BlockAdornments/FoldState remain unproduced by
design — no inlay/blame/lens/fold/diff source exists in pmacs yet.
Honest stubs (the M11.1 "declared, not yet wired" discipline), not
empty messages every frame.
Dispatcher updated for SemanticRenderState::new(frontend_id). Lib
(1394 crdt / 1239 non-crdt) + integration green on both feature
flavors; clippy -D warnings clean on both.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Before this, pmacs sent and consumed LSP `Position.character` as raw
UTF-8 byte offsets while advertising no `positionEncoding`, so per
the LSP spec every non-3.17 server assumed UTF-16. rust-analyzer is
lenient; clangd/gopls/pyright are not — any non-ASCII byte before a
position silently corrupted definition jumps, diagnostic spans,
hover targets, and formatting edits. This is the correctness gate
before Python/C/C++/Go.
- Negotiation: advertise `general.positionEncodings:["utf-8","utf-16"]`;
honour the server's `capabilities.positionEncoding`; default UTF-16
(LSP spec default) when absent. Stored per-LspClient.
- Codec: per-line byte<->utf-8/utf-16. UTF-8 is an identity fast
path; mid-surrogate / mid-codepoint positions clamp to the
containing char's start (a unit test caught and fixed an overshoot
in the first cut).
- Document cache: (server,uri)->text mirrored from did_open /
did_change_full, dropped on did_close and at every teardown site
(start_generation / on_exit / forget) alongside the existing drain.
- One conversion at the Rust boundary: a recursive JSON Position
rewriter at the two inbound seams (absorb_routed_response,
absorb_publish_diagnostics) plus outbound at request-build. Zero
store / Lua / consumer changes — completion popup, diagnostics
gutter, and lsp.lua all stay byte-uniform.
- Non-destructive fallback: a Position on a line absent from the
cached doc (cross-file / not-yet-opened) is left unconverted, not
collapsed to 0. Correct production behaviour (cross-file encoding
is v0.2, tied to deferred cross-file nav) and fixes the m4_12
definition tests.
Tests: 6 lib unit tests (negotiation default, utf-8 identity, utf-16
non-ASCII round-trip, astral surrogate pair, nth_line EOF semantics,
recursive rewrite). pmacs_fake_lsp `posecho` mode (advertises
utf-16, echoes the wire position into the result uri as `pos:N`);
end-to-end m4 test asserts both directions independently and
discriminatingly against a `é=x` fixture.
Gate: fmt clean; clippy --all-targets -D warnings clean; lib
1229/0; m4_acceptance 60/0; m9_1 18/0; m8_1/m8_9/m8_10 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the editor-blocking `poll_until` tick-loop in the LSP UX
with the M9.1 external-settle pattern: each `textDocument/*` request
registers a pending entry via `AsyncRuntime::register_external` and
returns the job id; the JSON-RPC response (or a server-teardown /
cancel / timeout) settles it, resuming a `Handle:await()` coroutine.
No worker thread is occupied for the round-trip.
Hybrid result delivery (operator decision): the response is absorbed
into the typed stores *and* carried through the Handle. The
completion popup and diagnostics gutter keep reading the stores
untouched; request/response command code awaits the value directly.
Core (src/lsp.rs):
- `LspManager` gains `runtime: SharedAsyncRuntime` (threaded through
`make_lsp_manager` / editor.rs, mirroring `make_mcp_manager`) plus
a `(server, request_id)` -> PendingExternal awaiter table parallel
to `pending_routes`.
- `request_*` return the async `JobId` (`= u64`, signature
unchanged; no caller consumed the old JSON-RPC id).
- `handle_response` settles every non-cancelled awaiter ok/failed
alongside store absorption; null result still wakes await with nil.
- Awaiters drain-cancelled at all three `pending_routes` purge sites
(restart generation flip / terminal exit / forget) so a coroutine
cannot park on a server that went away.
- Per-tick sweep: per-awaiter cancellation (Handle:cancel() or
supersede via a stable `lsp:{method}:{sid}:{uri}` key), with
`$/cancelRequest` + `cancelled_rids` on abandonment to drop the
cancel/response race silently. Mirrors mcp.rs.
- Per-request timeout (default 10s, `pmacs.lsp.set_request_timeout_ms`):
an alive-but-silent server fails the await instead of hanging.
Lua surface:
- `_request_*_raw` job-id bindings (mirror `pmacs.mcp._send_request_raw`).
- builtin/runtime/lsp.lua: Handle wrappers + the four commands
rewritten to spawn `pmacs.async` coroutines that `:await()`;
`poll_until` removed. Server-gone / error surface as structured
await failures in the modeline.
Tests:
- pmacs_fake_lsp: `error` / `silent` modes for deterministic
failure-path coverage.
- 5 end-to-end await-path tests (success+store, server-error->failed,
server-stop->cancelled, timeout->failed, supersede->cancelled).
Gate: fmt clean; clippy --all-targets -D warnings clean; lib
1223/0; m4_acceptance 58/0; m9_1_acceptance 18/0 (MCP unaffected).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of the CI Lint regression: commit 6113c53 bumped
rust-version 1.85 -> 1.95. clippy::collapsible_if is MSRV-gated —
collapsing `if let { if let }` needs let-chains, stabilized in Rust
1.95. At MSRV 1.85 clippy suppressed these; at 1.95 it emits them.
The patterns were pre-existing; the MSRV bump surfaced 47 of them
and turned `Lint (luajit)` / `Lint (lua54)` red at HEAD (was green
through PR #7; red from PR #8 = the release-prep MSRV bump).
Resolution (operator-chosen: autofix into let-chains): applied
`cargo clippy --fix` across the luajit, lua54, and crdt lanes
(--all-targets). The fix only applied with the lint at warn level;
`-- -D warnings` turns it into an error and blocks --fix.
Verified on the pinned 1.95.0, all three lanes:
clippy --all-targets -D warnings clean (luajit / lua54 / crdt);
fmt 0 diffs; lib tests 1223/0.
Note: the prior #6 "quiescent audit, clippy clean" was inaccurate —
clippy was not actually re-run there (build/version/fmt only), so
this MSRV-gated regression went uncaught until the live attach-debug
investigation surfaced it. This commit restores genuine clippy
cleanliness at MSRV 1.95.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Levi Neuwirth