Go to file
Levi Neuwirth db24abb64e
feat(lsp): D3 --- the file watcher stops sleeping and walks once per scan (#233)
Implements docs/lsp-file-watch-d3-framing.md revision 4, approved
2026-08-11 with the four rulings adopted as proposed: the honest bar
(absent at idle, one attributable job per concurrently due group), no
exclusions by default, server root_uri -> cwd -> attachment fallback,
and constants rather than config keys.

pmacs.fs.walk_tree: the whole recursive tree as ONE cancellable job
(JobKind::FsWalkTree, reply reuses ReplyKind::ReadDir --- identical
payload shape, so the Lua boundary needs no second conversion). Names
are base-relative; symlinks recorded, never traversed; an unreadable
subdirectory skips its subtree (scan_tree's pcall behaviour); only the
root failing to open fails the walk; the cancel token is polled once
per directory. Eight Rust unit tests, including flat-directory entry
parity with read_dir_blocking and the two review-round cancellation
cases (empty-tree pre-cancel; mid-walk via the cfg(test) entry hook).

The watcher itself is rewritten as the framed group scheduler. No
sleeps anywhere: one process.after-tick subscription (installed once
and guarded --- pmacs.hook.remove does not exist) drives every
(server, base) group's deadline off monotonic_ms, autosave's Q#AS2
idiom. The old design held one pool thread per sleeping watcher and
allocated 1 sleep + D read_dir jobs per watcher per tick --- 1,326
per tick for rust-analyzer's six watchers on this 220-directory
checkout. At idle there is now NO running job, which is also the
strongest witness in the suite: activity_summary settles to None, and
that assertion is unwritable under the old design.

The scheduler is the framing's state machine, all three review rounds
included: single-flight per group with generation-checked completions;
deadlines advanced from completion; the round-3 three-arm completion
partition (success / stale-or-retired / live non-success, with the
failure latch and quiet cancellation); joins wake the group, queue
exactly one follow-up mid-walk, and never reset the backoff curve;
per-watcher baselines --- the first snapshot whose WALK STARTED after
the join; membership captured at scan start; per-member cancellation
recheck at emit through the preserved _after_scan_for_tests seam;
backoff 250ms x2 to a 4s cap, reset by any emitted change; retirement
cancels the in-flight walk cooperatively.

Verification: eighteen acceptance tests. The six #234 tests are
byte-unchanged and green. Ten witnesses cover the framing's plan (the
review rounds added the fallback-determinism and root-boundary pair,
making twelve):
idle absence (and never a sleep purpose), one walk job per scan on a
twelve-directory fixture, join-wakes plus the registration epoch,
queued baseline for a mid-walk join (driven by saturating the worker
pool so the walk genuinely queues), single-flight under a withheld
completion pump, retirement and rebaseline through the fake's
unregister/re-register triggers, live cancel via pmacs._async._cancel
on the queued job, live failure with the once-per-error latch and the
preserved-snapshot recovery (DELETED for the pre-failure file is only
derivable from the retained snapshot), backoff shape from seam
timestamps, and the configured-root base.

Every witness was mutation-tested. Two findings from the bites:

- Retirement is DOUBLE-ENFORCED (unregister path and post-scan sweep)
  and biting either copy alone is masked by the other; only biting
  both goes red. Kept deliberately: the sweep covers seam-cancelled
  members, the unregister path covers idle groups whose next deadline
  is seconds away.
- The first idle probe was VACUOUS: it read pmacs.async instead of
  pmacs._async, errored, and the unwrap_or_default made every sample
  read as "absent". The probe now expects rather than defaults, so a
  broken probe is a red test, not a green lie.

One environmental fact, recorded in the lane: an empty stray /tmp/.git
(since removed) made project detection root every markerless tempdir
fixture at /tmp, which under Q#D3-3 the watcher then faithfully
watched. A markerless-fixture red that looks like a watcher bug may be
an ancestor marker.

A pre-commit review round found four blockers, all fixed here:

- walk_tree checked cancellation only inside its entry loops, which an
  EMPTY tree never enters --- a pre-cancelled queued walk returned an
  empty SUCCESS, which the success arm would commit and diff into a
  deletion storm. Cancellation is now checked before opening and
  before returning, cancellation outranks a missing-root error, and a
  unit test pins both.
- The neither-root-nor-cwd attachment fallback was still pairs-order
  nondeterministic --- the exact accident D3 set out to remove, behind
  a comment claiming otherwise. It now takes the lexicographically
  smallest attachment directory. Verified at the spawn sites: every
  server spawned with an attached file gets cwd = root, so the arm is
  defensive and unreachable through production spawning --- which is
  also why it carries no through-the-server witness.
- A base at the filesystem root joined as //path (and file:////path in
  URIs). Both join sites now go through join_under, the root-aware
  idiom dired's handler already uses, and the dir-of capture for a
  root-level file ("" from the match) normalizes to "/".
- The walk-count and scan-times probes defaulted on error, so two
  broken probes could compare equal and pass the retirement witness.
  Every probe now expects --- a broken probe is a red test, the same
  correction the vacuous idle probe forced.

A second pre-commit round found three more, all fixed here:

- Mid-walk cancellation was UNWITNESSED: both Rust cancel tests
  pre-cancelled and the acceptance test cancelled a queued walk, so
  deleting the internal polls left every test green. A cfg(test)
  entry hook now flips the token at an exact entry boundary and the
  witness asserts the walk stopped NEAR it (bound on entries
  processed), which is what discriminates the polls from the
  entry/exit checks. The retirement witness now holds a walk in
  flight across the unregister and asserts the job settles cancelled.
- The "unreachable fallback" claim was WRONG: pmacs.lsp.spawn may
  omit both cwd and root_uri, and ensure_server adopts such a live
  server for markerless files (root_uri and key_uri both nil). The
  lexicographic-minimum fallback now has a through-the-server
  witness: five sibling directories, the minimum opened last ---
  five, because with two the build's hash order coincided with the
  lexicographic answer and the first-pairs bite survived.
- The root-boundary joins gained a witness through exported
  production functions (the _deliver_status pattern): the matcher and
  URI builder driven at base "/", where reverting either join_under
  call makes the anchored glob refuse //hit and the URI grow a fourth
  slash. No fixture can walk / for real.

Verification totals after both rounds: eight walk_tree unit tests,
eighteen acceptance tests (six byte-unchanged, twelve witnesses), all
mutation-verified.

No wire change, no PROTOCOL_VERSION bump; walk_tree is an fs binding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 14:59:28 +02:00
.github/workflows feat(release): binaries on tag — Distribution Stage 1 2026-08-01 14:40:47 -04:00
audit V0.2-prerequisite pull-forward + M10.11 clean audit round 2026-05-18 10:31:31 -04:00
builtin feat(lsp): D3 --- the file watcher stops sleeping and walks once per scan (#233) 2026-08-11 14:59:28 +02:00
docs feat(lsp): D3 --- the file watcher stops sleeping and walks once per scan (#233) 2026-08-11 14:59:28 +02:00
pmacs-gpu feat(workers): a required purpose on every job and process — worker identity Stage 1 2026-08-10 14:51:54 +02:00
pmacs-protocol feat(discovery): M-x rows carry descriptions — protocol v22 -> v23 2026-08-10 13:52:32 +02:00
proptest-regressions M10.10 ship gate 2026-05-13 16:28:46 -04:00
scripts docs: record the witness as closed, and what the audit found next door 2026-08-09 18:18:34 +02:00
src feat(lsp): D3 --- the file watcher stops sleeping and walks once per scan (#233) 2026-08-11 14:59:28 +02:00
tests feat(lsp): D3 --- the file watcher stops sleeping and walks once per scan (#233) 2026-08-11 14:59:28 +02:00
.gitignore audit remediation: workspace clippy gate + stale metadata + cruft (F-001/F-013/F-015) 2026-07-03 10:31:17 -04:00
AGENTS.md docs: add COHERENCE.md as a required doc, audited against the codebase 2026-07-25 11:37:21 -04:00
CHANGELOG.md docs(changelog): remove dangling prerequisite links 2026-07-22 19:54:07 -04:00
CLAUDE.md docs: add COHERENCE.md as a required doc, audited against the codebase 2026-07-25 11:37:21 -04:00
COHERENCE.md Merge main into git-status-stage1 --- the prerequisite has landed 2026-08-10 21:53:55 +02:00
Cargo.lock feat(release): binaries on tag — Distribution Stage 1 2026-08-01 14:40:47 -04:00
Cargo.toml feat(release): binaries on tag — Distribution Stage 1 2026-08-01 14:40:47 -04:00
LICENSE-APACHE Initial commit: v0.1.0 2026-05-03 19:51:06 -04:00
LICENSE-MIT Initial commit: v0.1.0 2026-05-03 19:51:06 -04:00
README.md docs: absorb the v1.1.0 release, and correct what it made stale 2026-08-01 18:09:59 -04:00
TEST_IMPROVEMENT.md review round 2: arm the required-checks name-coupling trap, restore m6 2026-07-29 14:16:27 -04:00
build.rs Initial commit: v0.1.0 2026-05-03 19:51:06 -04:00
rust-toolchain.toml rust-toolchain.toml: add rust-analyzer component (pin regression fix) 2026-05-18 11:58:46 -04:00
rustfmt.toml Initial commit: v0.1.0 2026-05-03 19:51:06 -04:00

README.md

Pmacs

Parallel Emacs --- a Rust-cored, Lua-scripted editor in the Emacs tradition.

Pmacs runs the editor's hot path (rope, buffers, views, async runtime, process supervision) in Rust, and exposes the rest --- commands, keymaps, hooks, packages --- through an embedded Lua VM. The design follows Emacs in shape (configurable, introspectable, programmable from inside) but discards the single-threaded substrate; workers, message bus, and a coroutine-based async surface are core primitives, not bolt-ons.

The editor is partitioned into a long-lived instance (the daemon that owns buffers, processes, and language services) and thin frontends that attach over a typed protocol (currently v20). Two frontends ship today:

  • a TUI (crossterm cell grid), attachable locally over a Unix socket or remotely over SSH, with reconnect-on-drop modeled on mosh; and
  • pmacs-gpu, a GPU frontend (wgpu + winit + glyphon) that renders from a semantic projection of editor state --- style spans, decorations, inlay adornments --- rather than a character grid, and edits optimistically against a local CRDT replica for latency-free typing.

Buffers are optionally CRDT-backed (loro, behind --features crdt), so multiple frontends --- TUI and GPU, local and remote --- can edit the same buffers concurrently with live cursor/selection presence.

Status

v1.1.0 --- stable core, active development, and the first release with prebuilt binaries. The v1.0 gate (the instance/frontend partition, the Lua surface, and a REPL package audited to use zero direct Rust core access) shipped some time ago. Development since has expanded the semantic frontend protocol from v6 through v21, brought the GPU frontend near input/render parity with the TUI, and completed the LSP, editing, persistence, themes, and terminal arcs. Recent work added major modes and modeline detection, a typed configuration registry, composable statuslines, multi-language syntax processing, cross-frontend tab-width parity, a directory browser, a describe/list command family, and a bottom panel on both frontends.

Current direction lives in COHERENCE.md (the product-coherence thesis and its audited priority order) and docs/agent-handoff.md (durable project state). docs/roadmap-2026-07.md is a historical planning snapshot and is no longer the authority.

Public contributions are open: use, evaluate, file issues, and send pull requests.

Highlights

Editing & UI. CUA-style region editing plus Emacs kill/yank and kill-ring bindings; linear undo/redo; query-replace; incremental substring and regex search (C-s / C-r / C-M-s); comment, auto-indent, auto-pair, transpose, case, line, and region operations; line-number gutter with absolute, relative, and hybrid modes; diagnostic signs; context menu; OS clipboard integration (OSC 52 in the TUI, native in the GPU); minibuffer completion with persisted history; buffer-list and compilation modes; self-navigable help. Named ui.* theme faces, live GPU font selection, and composable per-window statusline providers keep chrome and modelines runtime-configurable. Saves are atomic (temp + rename + parent fsync, mode-preserving).

Language intelligence. The async LSP client provides diagnostics, rename with prepareRename, cross-file definitions, hover, signature help, references, document symbols, code actions, formatting, semantic tokens, and inline inlay hints. Preconfigured servers cover Rust, C/C++, Python, Go, JavaScript/TypeScript, Lua, Bash, TOML, Zig, Dockerfile, CMake, JSON, and YAML. Bundled tree-sitter grammars include those languages plus Markdown, Make, and CUDA; nested Markdown fences and frontmatter use multi-language injections, and locals-query processing distinguishes shadowed builtins. Bounded Emacs and Vim modelines join extensions, exact filenames, and shebangs in one fresh-load language decision. That decision initializes the buffer's major mode, drives syntax/LSP/pairing/comment behavior, and enables mode-scoped keymaps. A persistent project-symbol index (.pmacs/index.json) rides the same worker infrastructure.

Collaboration & frontends. With --features crdt, buffers are CRDT-backed and any number of frontends attach to one daemon and edit concurrently; peers see each other's cursors and selections as translucent washes. The TUI and GPU frontends both host owned full-screen terminal sessions; protocol-v19 terminal frames preserve the fixed-cell VT screen while each frontend owns its scroll/selection/input context. The GPU frontend also provides a live minimap, wavy diagnostic squiggles, a status band, and optimistic local editing that rebases in-flight edits through authoritative frames. Buffer text, syntax, diagnostics, carets, hits, and minimap geometry now share one eight-column tab projection without mutating source bytes.

Extensibility. The pmacs.* Lua namespaces cover buffers, windows, commands, global/mode/buffer keymaps, hooks, themes, statusline providers, tree-sitter, LSP stores, async workers, and a PTY-aware process supervisor. The typed, introspectable pmacs.config registry supports global and buffer-local values, listeners, startup-only settings, and describe-setting. A package manager installs from git (github:owner/repo, version/branch/commit pins) with transitive dependency resolution and a SHA-256 lockfile. Pmacs is also an MCP client: packages can spawn MCP servers and consume their tools, resources, and prompts --- AI integrations are packages over a transport, not a built-in feature. The bundled REPL package is written entirely against the public Lua API.

Running

Single-process TUI:

pmacs [FILE]                 # TUI; -nw reserved for when a GUI default lands

GPU frontend (one command; the root binary starts or reuses the daemon):

pmacs --gpu                         # default instance; no initial file
pmacs --gpu README.md               # default instance; open one file
pmacs --gpu --socket NAME FILE      # named instance; bare NAME →
                                    #   <runtime>/pmacs/NAME.sock
pmacs --gpu -- --leading-dash       # `--` ends option parsing

pmacs --gpu requires the root pmacs binary to be built with the crdt feature. It discovers a sibling pmacs-gpu binary first, then falls back to pmacs-gpu on PATH. When FILE is present, the daemon loads or creates it and completes startup hooks before the GPU window appears. Closing the window detaches only that frontend; the daemon remains available for later GPU or TUI attaches.

Daemon + attached TUI frontends:

pmacs --daemon --socket NAME        # foreground daemon
pmacs --attach --socket NAME        # TUI frontend; F12 detaches
pmacs --attach user@host            # remote TUI over SSH

For debugging an already-running daemon, the low-level GPU command stays available and never auto-starts or replaces anything:

pmacs-gpu --attach /absolute/path/to/pmacs.sock

pmacs --attach also understands ssh:user@host/instance, local:/path.sock, and bare hostnames (treated as SSH). See pmacs --help for the full matrix.

User configuration is plain Lua at $XDG_CONFIG_HOME/pmacs/init.lua (default ~/.config/pmacs/init.lua), loaded after the builtin runtime so plain assignments override defaults --- keybindings, pmacs.lsp.config, theme overrides, and package installs all live there.

Install

Download an archive from the releases page, unpack it, and put both binaries somewhere on your PATH.

Keep pmacs and pmacs-gpu together. pmacs --gpu looks for pmacs-gpu beside itself first and only then falls back to a PATH lookup, so an unpacked release is self-contained as long as the two stay in the same directory.

Verify a download:

sha256sum -c SHA256SUMS --ignore-missing
platform built on notes
Linux x86_64 Ubuntu 22.04 requires glibc ≥ 2.35 — Ubuntu 22.04+, Debian 12+. RHEL 9 (glibc 2.34) is not supported yet.
macOS arm64 macOS 15 Apple Silicon only; Intel is not built yet. Binaries are unsigned and not notarized, so Gatekeeper will quarantine them until you allow them explicitly.

Releases carry binaries only — there is no in-place update, rollback, or package-manager distribution yet. Build from source for any platform not listed, and see "Runtime dependencies" below for what the editor assumes is present.

Build

Builds on the toolchain pinned in rust-toolchain.toml (Rust 1.95.0, edition 2024); rustup selects it automatically.

# Coherent root + GPU release build. The package-qualified feature keeps
# the separate pmacs-gpu package feature-free while enabling CRDT in pmacs.
cargo build --release --workspace --features pmacs/crdt

target/release/pmacs --gpu README.md # one-command managed GPU file launch
cargo run --release -- --version     # default-run selects the pmacs binary
cargo test --workspace              # unit + integration tests (all crates)
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings   # incl. pmacs-gpu

Feature matrix

Cargo features fall into two independent axes. Do not use --all-features — it enables both Lua flavors at once, which cannot build (see below).

Feature Axis Notes
luajit Lua flavor Default. LuaJIT backend via mlua (vendored).
lua54 Lua flavor Lua 5.4 fallback for hosts without LuaJIT (big-endian, …).
crdt Buffer Opt-in CRDT-backed buffer mode (adds the loro dep). v1.0 builds enable it; orthogonal to the flavor.

Exactly one Lua flavor must be enabled — luajit or lua54, never both (and never neither). They map to mlua's mutually-exclusive Lua backends, so --all-features (or --features luajit,lua54, or --no-default-features with no flavor) fails in the mlua-sys build script with "You can enable only one of the features: …". That check lives in a dependency cargo builds first, so pmacs can't replace it with a friendlier error — the fix is to build a specific flavor. Supported build lines:

cargo build --release                                   # luajit (default)
cargo build --release --no-default-features --features lua54
cargo build --release --features crdt                   # luajit + crdt
cargo build --release --no-default-features --features lua54,crdt

CI, cargo hack, and distro tooling should iterate the flavors explicitly (--no-default-features --features <flavor>[,crdt]) rather than reaching for --all-features. Both flavors pass the full test suite; CI runs the matrix on every push.

Release-only perf gates (M5 keystroke-to-render, M6 ingest/RSS/cancel and scrollback navigation/search) are #[ignore]'d during normal test runs and exercised in CI under dedicated jobs. The GPU frontend has headless render tests (offscreen wgpu, pixels read back) that run in CI under lavapipe and skip gracefully on machines without a Vulkan adapter (PMACS_REQUIRE_GPU=1 turns a missing adapter into a hard failure).

Runtime requirements

The pmacs binary depends on a small set of POSIX command-line tools at runtime. The dependency exists because the project enforces #![forbid(unsafe_code)] everywhere, including in tests; calls that would otherwise need unsafe (PTY raw-mode setup, signal name translation) are routed through trampolines that exec these tools.

  • /bin/sh (POSIX shell). Used for the PTY raw-mode trampoline: /bin/sh -c 'stty raw -echo </dev/tty 2>/dev/null; exec "$@"' -- configures the controlling TTY's line discipline before exec'ing the actual subprocess. Required by the REPL package and any other caller that spawns a process in raw PTY mode.

  • stty (coreutils). The line-discipline configurator invoked by the trampoline above.

  • coreutils more broadly. The M6 process-supervisor tests spawn cat, yes, and which; absent these the test suite (not the editor itself) degrades. which is also used by the M6.5 shell-locator helper to find bash / zsh / fish for per-shell integration tests. The M7.2 fetcher's timeout test uses sleep.

  • setsid (util-linux, Linux only, optional). The process teardown-deadlock test uses setsid --fork to orphan a grandchild, which is the only way to reproduce that deadlock without depending on shell & semantics (they differ between bash and dash). The test skips when setsid is absent, so a minimal or BusyBox environment still runs cargo test --lib; set PMACS_REQUIRE_SETSID=1 to make that skip a failure, as CI does on Linux.

  • /bin/bash (optional, Linux only). The signal diagnostic's job-control corroboration test needs a terminal whose foreground process group is not the spawned leader, which bash -m produces by running a foreground job in its own process group. The path matters: the test spawns /bin/bash directly rather than resolving bash on PATH, and skips when that path is absent. Set PMACS_REQUIRE_BASH=1 to make the skip a failure, as CI does on Linux.

    It is deliberately not armed on macOS, which ships bash 3.2 but where a non-interactive bash -m was measured in CI to keep the terminal on the leader — so the divergence the test needs never happens there. The divergent case is pinned on every platform by injecting the foreground group instead.

  • git (added in M7.2). Required for any package operation: the package fetcher shells out to git to clone, fetch, and resolve refs, with a deterministic environment (GIT_TERMINAL_PROMPT=0, GIT_CONFIG_NOSYSTEM=1, LC_ALL=C, inherited GIT_* variables stripped). Authentication for private repositories rides the user's existing git configuration (credential helpers, SSH agent), so packagers do not need a separate auth story. Pre-M7 builds without package operations do not need git.

  • tar (added in M7.3). Required for pmacs.packages.install: the installer materializes a snapshot via git archive --format=tar piped into tar -x -C <dest>, which keeps the on-disk install directory self-contained (no .git linkage back to the bare cache, no working-tree state). GNU tar and bsdtar both work. Pre-M7 builds and any path that doesn't call pmacs.packages.install{...} do not need tar.

Distribution packagers should ensure these are runtime dependencies of the pmacs package. On a typical Linux distribution, busybox or GNU coreutils plus a shell of any kind satisfies the requirement; on macOS the system shell and /usr/bin/stty are both standard.

The Lua VM (LuaJIT or Lua 5.4) is statically vendored via mlua's vendored feature, so there is no external Lua dependency at runtime.

The GPU frontend additionally needs a Vulkan-capable driver stack (any real GPU driver, or lavapipe for software rendering); its font (JetBrains Mono, OFL-licensed) is bundled into the binary.

Layout

The workspace has three first-party crates:

src/                 pmacs — the core + TUI + daemon
  rope.rs              persistent byte-sequence backing every buffer
  buffer.rs            buffer + view chain + undo/redo
  editor_core.rs       cursor + commands + edit dispatch
  crdt.rs              loro-backed CRDT buffer state (feature `crdt`)
  daemon.rs            instance side of the frontend partition
  attach.rs            frontend side; transports + reconnect
  semantic_render.rs   semantic-frame producer (StyleSpans, Decorations, …)
  lsp.rs               language-server client
  diag.rs, highlight.rs  diagnostic + syntax/semantic-token rendering
  syntax.rs            tree-sitter integration
  search.rs            incremental search (substring + regex)
  minibuffer.rs        prompt, completion, persisted history
  menu.rs              context-menu model
  file_io.rs           atomic saves + external-modification detection
  async_runtime.rs     worker pool + message bus
  process.rs           PTY-aware process supervisor
  ansi.rs              ECMA-48 parser
  project.rs, project_index.rs  project detection + symbol index
  packages/            resolver, fetcher, installer, lockfile, loader
  mcp.rs               MCP client (packages speak to MCP servers)
  lua_bindings/        pmacs.* Lua surface installers
  text_view.rs         cell-grid renderer
  frontend.rs          crossterm TUI
  main.rs              entry point (TUI / daemon / attach modes)

pmacs-protocol/      wire types + framing codec shared by all frontends
pmacs-gpu/           the GPU frontend (wgpu + winit + glyphon)

builtin/             Lua runtime shipped with the binary
  commands/default.lua  named commands for every editor primitive
  keymaps/default.lua   default key bindings
  hooks/default.lua     built-in hook definitions
  menus/default.lua     context-menu items
  runtime/              async, lsp, syntax, mcp, fs runtimes
  packages/repl/        the bundled REPL package

docs/                design notes, framing docs, and the roadmap
tests/               integration tests (acceptance gates per milestone)

License

Dual-licensed under either of:

at your option.