pmacs/docs/lsp-file-watch-d3-framing.md

LSP file watcher D3 — the polling cost — framing

Status: revision 4 — DRAFT, review corrections absorbed; awaiting the user rulings Q#D3-1..4. No implementation may begin from this document.

Continues issue #233, which stays open until this lane closes it. D1 and D2 — matching correctness and the re-registration leak — merged as #234 (ae84d58); this frames the remainder the user ruled next on 2026-08-11: the watcher is now correct and still walks everything, every tick, forever.

Review round 1 — five findings, and what each changed

Revision 1 was reviewed 2026-08-11 and did not survive it. Recorded here because three of the five are cases where the framing reasoned from the wrong mechanism, which is exactly what a framing review exists to catch before code does.

• P1 — the promised idle state was impossible. Revision 1 promised "one brief walk_tree blip every four seconds"; but pmacs.workers.sleep allocates a running job for its whole duration (dispatch_sleep, src/async_runtime.rs:1022-1037 — the job sleeps in 1 ms slices on a pool thread), and activity_summary counts every Running job (src/async_runtime.rs:1570). A 4 s backoff sleep would render as a constant ⋯1 sleep 4000ms, and filtering sleeps from the indicator would touch the instrument this lane declares out of scope. Revision 2 removed the sleep from the design entirely (see "The cadence" below) — the fix is the codebase's own idiom, not a new mechanism.
• P1 — the scan root must be the server's, not a freshly detected one. Revision 1 proposed pmacs.project.detect for the string-form base. Server rooting honours configured strings and custom resolvers first (project_root_for, builtin/runtime/lsp.lua:783), and the bundled texlab entry documents why its root cannot be project.detect (Q#LX2, lsp.lua:284 — texlab wants the document root, not the repository root). The server's own root_uri and cwd are already exposed (pmacs.lsp.list, src/lua_bindings/mod.rs:10985-11002; root_uri is the spec field verbatim, nil when the server never asked for a root). Revision 2 rooted the scan at what the registering server actually serves: root_uri → cwd → attached-file directory, in that order.
• P1 — coalescing needs registration-epoch semantics. Revision 1 said "route the shared diff through every watcher" without defining which watcher set owns a diff. A watcher registered between two snapshots would receive a false CREATED for a file that predates its registration; membership changing during a walk recreates either #234's stale-watcher batch or the same pre-registration event. Revision 2 defined shared snapshots with per-watcher baselines, refined by round 2 below.
• P1 — "VCS-only exclusion is safest" was wrong. A hard skip silently ignores a server that legitimately registers .git/HEAD or **/.git/**, and glob semantics mean even **/*.rs can match under .git/ — so any unconditional exclusion is a deviation from the registered contract, not a safe default. Revision 1 also overweighted the win: exclusion was a job-count lever when every directory was a separate job, and the walk primitive removes that economics. Revision 2 defaults to no unconditional exclusion.
• P2 — the arithmetic used the issue's machine, not this one. With D = 220 on this checkout it is 220 read_dir jobs plus one sleep per watcher per tick — 221, or 1,326 across rust-analyzer's six watchers — and revision 1's proposed steady state was itself two jobs (sleep + walk), not one. Corrected throughout.

Review round 2 — the scheduler was underspecified

Round 2 (2026-08-11) closed the round-1 findings and found the cadence's own semantics missing: revision 2 said when scans become due but not what happens when due-ness, in-flight walks, joins, and retirement collide.

• P1 — a joining watcher must force an immediate baseline scan. Revision 2's baseline was "the first snapshot completed after join" — but a backed-off group's next snapshot can be 4 s away, so a file created after registration and before that delayed snapshot would fold into the baseline and never be reported. Today, registration begins its initial scan immediately (lsp.lua:2074); the coalesced design must preserve that. A join pulls the group's next_scan_at to now; if a walk is already in flight, exactly one immediate follow-up scan is queued. The baseline is sharpened to match: a snapshot serves as a watcher's baseline only if its walk started after the join — an in-flight walk may have passed a directory before a pre-join file appeared there, and using its snapshot as a baseline would turn that file into a false CREATED on the next diff.
• P1 — single-flight and retirement were undefined. The document itself establishes that walks can outlive their interval, so "every due group starts a scan" permits overlapping walks — restoring multiple jobs, completing snapshots out of order, and making the epoch state ambiguous. The group scheduler below is now a defined state machine: one in-flight scan per group, deadlines advanced from completion, stale completions rejected, and retirement (last member gone, or server death) that cooperatively cancels the walk — which puts cancellation into walk_tree's contract and tests.
• P2 — ⋯1 was an overclaim. Groups are keyed by (server, base), so several can be due on the same frame, and per-group single-flight still permits ⋯N. The bar is restated accurately: absence at idle; while scans run, one attributable job per concurrently due group. Global serialization is offered as the alternative under Q#D3-1 if ⋯1 must be guaranteed.

Review round 3 — live non-success closes the state machine

Round 3 (2026-08-11) accepted all three round-2 corrections and found one transition still absent: revision 3 specified successful and stale scan completions, but not a cancellation or failure while the group itself remains live.

• P1 — a live cancelled/failed walk needs a terminal transition. Every job is user-cancellable from *workers*, and Handle:await() raises structured cancelled and failed outcomes. Without a caught non-success path, the group can retain in_flight = true forever and silently stop watching. Revision 4 partitions completion into success, stale/retired, and live non-success (below). A live cancellation or failure commits no snapshot or epoch, emits nothing, preserves the previous snapshot and backoff curve, clears in-flight, and either serves a queued join immediately or schedules the next attempt normally. Failures are reported visibly and deduplicated per group; cancellation is an intentional user/runtime outcome and stays quiet.
• P2 — the queued-baseline latency bound was one walk short. A join just after a walk starts waits for the remainder of that walk and its own follow-up walk before the baseline snapshot completes. Revision 4 states the two boundaries separately: the baseline walk starts when the current walk completes, and its snapshot completes after that follow-up walk — never after the backoff cap.

Verified against the tree at add0ba1

Every claim below was read or measured this session (revisions 2–4 re-verified their corrections against the code).

• Each registered watcher is its own coroutine looping sleep(FILE_WATCH_INTERVAL_MS) → scan_tree (lsp.lua:1924, :2074-2083); the interval is 250 ms. Registration's initial scan runs immediately (:2074-2075).
• scan_tree awaits pmacs.fs.read_dir once per directory (:2038-2041), one async job each. walk recurses unconditionally; matches gates only whether an entry is recorded.
• A sleep is a pool-occupying running job. dispatch_sleep dispatches run_sleep onto the worker pool (async_runtime.rs:1022-1037), so every sleeping watcher holds one of the pool's available_parallelism - 1 threads for the full interval — the hazard autosave.lua:133-139 documents in so many words. Twelve pre-#234 rust-analyzer watchers were twelve mostly-sleeping pool threads.
• Jobs per tick per watcher = 1 sleep + D read_dirs. On this checkout D = 220, so 221 per watcher and 1,326 per tick for rust-analyzer's six watchers.
• 250 ms is a floor, not a period. The awaits are sequential, so a tree whose walk takes longer than the interval makes the effective period scan-bound — the loop never idles. The issue measured ~270 ms on a two-directory tree; D round trips dominate on real trees.
• Measured on this checkout: 220 directories, of which .git is 177 — over 80 % of the walk. (The issue's machine carried an in-tree target/: 187 directories, 46 outside .git+target. This machine exports an external CARGO_TARGET_DIR and still pays .git.)
• The string-form base is nondeterministic, found while framing: resolve_watcher's string arm takes the directory of the FIRST attachment pairs() happens to yield (:2150-2173) — table order, not a chosen root. #234 made matching correct per base; which base is still accidental.
• The codebase already has a no-job cadence idiom with five adopters. process.after-tick fires every frame, including idle frames (the run loops tick on a frame timeout), and pmacs.editor.monotonic_ms is the clock built for exactly such loops (lua_bindings/mod.rs:13833). autosave.lua's Q#AS2 sweep is the model: one clock read and one compare per frame, no job, no pool thread.
• pmacs.hook.remove does not exist (the P3 prerequisite gap, docs/agent-handoff.md §1a) — an after-tick subscription is permanent, so the scheduler installs once and early-returns when it owns no groups, exactly as autosave's does when disabled.
• Cooperative cancellation is the established job shape: every job body in async_runtime.rs polls cancel.is_cancelled() at its work boundaries (run_sleep per slice, the others per unit); walk_tree polling between directory reads inherits the pattern.
• No notify/inotify dependency in the tree and no ignore-list infrastructure to reuse — both re-verified, both carried from the D1/D2 framing.

What §9 asks of this lane

Not "quiet the modeline." The indicator is the instrument that found this, and the churn it shows is real; quieting it is explicitly out of bounds. The lane's job is to make the background work small, attributable, and honest: at idle there should be no running background work to report, and while scans run each should be one job named for its root.

The cadence — after-tick deadlines, not sleeps (round 1)

The per-watcher sleep loop is replaced by the Q#AS2 idiom: one process.after-tick subscription owns every scan group's schedule. Per frame it reads pmacs.editor.monotonic_ms once and compares each group's next_scan_at. Waiting allocates no job and no pool thread and renders no indicator segment — activity_summary returns None at zero by contract. While a scan runs, the indicator honestly shows its job.

The subscription installs once at module load and early-returns when no groups exist — it cannot be removed, because pmacs.hook.remove does not exist, and a guard is the house answer (autosave's enabled check).

The group state machine (rounds 2 and 3)

Per group — keyed (server, base) — the scheduler holds next_scan_at, the backoff interval, an in-flight record (handle, generation, and start time), a scan generation counter, a rescan_queued bit, and the last reported walk failure for deduplication.

• Single-flight. The after-tick check skips a group whose walk is in flight; a group cannot become due against itself. Concurrent walks, out-of-order snapshots, and ambiguous epochs are therefore unrepresentable, not merely avoided.
• Deadlines advance from completion. On scan completion, next_scan_at = completion time + current interval. A walk that outlives its interval degrades to back-to-back scans with a full interval between them — never to overlap.
• Stale completions are rejected. Each scan carries its group's generation at start; a completion whose group is retired, or whose generation is not the group's current one, is dropped before any state write or emit — #234's P2 recheck, applied at group scope.
• Current completions have three disjoint outcomes (round 3). The coroutine catches Handle:await() so its structured outcome cannot bypass group cleanup. A successful current completion clears in-flight, commits the snapshot and epoch, routes its diff, updates the backoff curve, and clears the failure-dedup latch. A cancelled or failed completion while the group is still live also clears in-flight, but commits no snapshot or epoch, emits nothing, and preserves the previous snapshot and backoff interval. It consumes rescan_queued by starting exactly one immediate scan; otherwise it sets next_scan_at = completion time + current interval. Failure is surfaced through the existing LSP status/error reporting shape once per distinct (group, error) until a success; cancellation stays quiet because workers.cancel-at-point makes it an intentional outcome. Retirement is the third arm: its cancelled completion is stale by construction and reaches none of this live state.
• Joins wake the group. A watcher joining sets next_scan_at = now. If a walk is in flight, rescan_queued is set instead, and completion of the current walk starts exactly one immediate follow-up scan. The joiner's baseline is the first snapshot whose walk started after its join (see below), so the baseline walk starts as soon as the current walk completes and its snapshot completes after that one follow-up walk — never a backoff cap away. The join-triggered scan does not reset the backoff curve; only observed changes do.
• Retirement. When the last member leaves (unregistration, or supersession with no successor) or the server dies, the group retires: the in-flight walk's job is cancelled cooperatively, its completion is rejected by the generation rule, and the group's schedule entry and snapshots are dropped. A re-registration that replaces members keeps the group alive — the superseded members are cancelled per #234's D2 and the new members join as above.

The scan root (round 1)

For a string-form (bare *.txt / absolute) registration the base becomes, in order: the server's root_uri (spec verbatim — nil when the server never asked for a root), the server's cwd, and only then the attached-file directory. These describe the workspace the registering server actually serves — including configured roots and custom resolvers like texlab's, which pmacs.project.detect can never reproduce (Q#LX2). This replaces the pairs()-order accident with a deterministic, server-owned answer. RelativePatterns keep their own baseUri, unchanged.

Coalescing, with registration epochs (rounds 1 and 2)

One scan group per (server, base). The group's scanner records all files (the matcher moves from scan time to diff time); each completed scan increments the group's snapshot epoch.

Delivery semantics:

• Each watcher's baseline is the first snapshot whose walk started after it joined — an in-flight walk may have passed a directory before a pre-join file appeared there, so its snapshot cannot serve as a baseline (round 2). A watcher receives diffs only between snapshots at or after its baseline. A file created after the group's previous snapshot but before a watcher joined therefore produces no event for that watcher — folded into its baseline, exactly as the initial scan folds pre-existing files today.
• Membership for delivery is captured at scan start; a watcher joining mid-walk waits for its queued baseline scan.
• Cancellation is rechecked per watcher at emit time — #234's P2 rule per member: a watcher superseded or unregistered during the walk emits nothing, and its replacement has no baseline yet, so it emits nothing either.
• Changes passing a watcher's matcher and kind mask are deduped by (uri, type) into the server's single workspace/didChangeWatchedFiles notification, as today.

The walk primitive

pmacs.fs.walk_tree(base) — the whole recursive walk as one job instead of one per directory: 220 read_dir jobs per scan on this repo become 1. The indicator shows one purpose (walk_tree <root>). An additive fs binding plus its async-runtime job; no wire change (fs bindings are not the frontend protocol) and no new crate. Two contract clauses, each with its own Rust tests:

• Symlinks are recorded, not traversed — scan_tree's loop-safety, preserved.
• Cancellation is cooperative and prompt: the job polls its cancel token between directory reads (the established async_runtime.rs job shape), so group retirement mid-walk stops the walk instead of orphaning it.

Exclusions (round 1) — none by default

Glob semantics make any unconditional skip a contract deviation: **/*.rs compiles with a separator-spanning prefix, so it can match under .git/, and a server may register .git/HEAD outright (branch-watching tools do). The only semantics-preserving default is no unconditional exclusion, and with the walk primitive the economics support it: exclusion was worth 80 % of the job count when every directory was a job; inside one walk_tree job it is only readdir syscalls, and the whole 220-directory walk is a few milliseconds of one pool thread per scan.

The option space, for Q#D3-2: (a) no unconditional exclusion — the proposed default; (b) opt-in exclusion through configuration, for users with pathological trees, framed explicitly as a watcher-contract trade; (c) matcher-aware pruning — skip a subtree only when no active watcher's pattern can match under it — sound but almost never fires against real registrations, because **/-leading globs can match anywhere; (d) a hard built-in VCS skip, which revision 1 called "safest" and is not: it is (b) without the opt-in.

Idle backoff

The interval doubles while consecutive scans observe no change, capped at 4 s; any change batch resets it to 250 ms. Under the after-tick cadence a longer interval costs nothing while waiting — backoff bounds scan frequency, not sleep-job length. Worst-case latency for an external change at idle equals the cap except at registration, where the join rule forces an immediate baseline (round 2). LSP imposes no latency bound, and edits made through pmacs never depended on the watcher (the server sees didChange). The watcher exists for git checkouts, generated files, and other editors.

Deliberately staged separately — kernel notification

notify (inotify / FSEvents / kqueue) eliminates polling. Also: a new dependency, a new Rust subsystem, a Lua binding, a platform matrix, and an interaction with §9's ownership model. Staged as its own framing — not because it is wrong but because everything above is a pure win it does not obsolete (a kernel watcher still needs the initial scan and a polling fallback), and a new-crate decision deserves its own review.

Proposed shape — Stage 1

After-tick cadence with the group state machine + walk primitive + coalescing-with-epochs + backoff; no exclusions by default; server-owned scan root.

At rest on this repo with rust-analyzer attached: from 1,326 jobs per scan-bound tick (six of them pool-thread-holding sleeps) to zero jobs at idle, with one walk_tree job per group for the few milliseconds each scan actually runs — at most every 250 ms under activity and every 4 s at rest, immediately once at registration.

Open rulings — each blocks implementation

• Q#D3-1 — the acceptance bar, stated accurately (round 2). At idle the indicator is absent (no running job exists — activity_summary's None-at-zero contract). While scans run it shows one attributable job per concurrently due group — ⋯N when N (server, base) groups are due on the same frame, each named for its root; a typical single-project session has one group. Alternative if ⋯1 must be guaranteed: a global scan queue serializing walks across groups, at the cost of coupling one server's scan latency to another's tree size. Which bar?
• Q#D3-2 — exclusions. Proposed: none by default, with opt-in exclusion as a documented contract trade (option b) if a user asks. Confirm, or rule for one of (b)/(c)/(d) above.
• Q#D3-3 — the scan root. Proposed: server root_uri → server cwd → attached-file directory. This widens the watched tree for servers with a real root (today it is one attached file's directory, chosen by hash order) — a behavioural change to a path real servers exercise. Confirm the order, or rule otherwise.
• Q#D3-4 — interval, cap, and backoff curve: constants or config keys. The D1/D2 framing refused a knob for a defect; with D3 the cadence becomes a designed mechanism, so keys are defensible — but more registry surface is coherence cost. Proposed: constants until someone asks.

Verification sketch

• Idle witness: with a server attached, watchers registered, and no file activity, activity_summary settles to None (the absent segment) between scans — the strongest form of the job-count claim, and unwritable under the sleep design.
• Scan-cost witness: one scan allocates O(1) jobs, not O(directories), on a tree with enough directories to discriminate.
• Join-wakes witness (round 2): with a group backed off at the cap, register a new watcher — a scan starts immediately (timestamps through the group seam); after that baseline settles, a newly created file is reported to the joiner.
• No-overlap witness (round 2): with a walk deliberately held in flight past its interval (through the group seam or by withholding the completion pump), the group allocates no second walk job; deadlines resume from completion.
• Retirement witness (round 2): the last member unregisters mid-walk — the walk's job is cancelled, its completion is rejected (no emit, no state write), and the group's schedule entry is gone; a server death takes the same path.
• Live-cancel witness (round 3): cancel an in-flight walk while members remain (the workers.cancel-at-point outcome) — no snapshot or epoch commits, the previous snapshot remains, in-flight clears, and the group scans again on schedule; repeat with a queued join and its baseline scan starts immediately.
• Live-failure witness (round 3): force a walk failure through the group completion seam — no snapshot or epoch commits, the prior snapshot remains, the group retries, one distinct failure is reported once, and a later success clears the dedup latch.
• Queued-baseline witness (round 2): a watcher joining mid-walk gets exactly one immediate follow-up scan, and its baseline is that scan, not the walk that was in flight at join.
• Epoch witness (registration between snapshots): create a file after the group's snapshot, then register a second watcher, then let a scan complete — the old watcher receives CREATED, the new one receives nothing for that file, and does receive events for files created after its baseline.
• Backoff witness: quiet scans lengthen the gap between scans and one change resets it — observed through scan timestamps at the seam, not through sleep purposes (there are none).
• Root witness: a server with a configured root watches that root, not the attached file's directory; texlab's resolver shape is the fixture model.
• Contract preservation: all six existing m4_24 watcher tests stay byte-unchanged and green.
• walk_tree Rust unit tests: symlinks recorded-not-traversed, cooperative cancellation observed mid-walk, signature parity with the Lua walk it replaces.
• Each new behaviour is mutation-tested against the defect it guards.

Coherence impact (§20)

• Journey steps: none added; step 5 is unchanged.
• Interaction islands: none.
• Config registry: none by default; Q#D3-2/Q#D3-4 could add keys and are flagged as such.
• Background-work attribution (§9): at idle there is genuinely no running background work, and the indicator's absence is then a true statement rather than a filtered one; each scan is one job named for its root. The ownership model remains §9 Stage 2's work. As a side effect the watcher stops holding pool threads while waiting.

Gates

./scripts/gate --acceptance m4_acceptance. No --protocol — no wire change, no PROTOCOL_VERSION bump; walk_tree is an fs binding, not a protocol message.