diff --git a/AGENTS.md b/AGENTS.md index 58c28f9..823b6e5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,11 +1,15 @@ # pmacs agent instructions -**Start here: read `docs/agent-handoff.md`, then +**Start here: read `docs/agent-handoff.md`, then `COHERENCE.md`, then `docs/active-work.md`, before taking on any work.** The handoff carries durable project state, working method, substrate invariants, and the -standing backlog. The active-work ledger carries volatile branches, +standing backlog. `COHERENCE.md` carries the product-coherence thesis +and its audited ground truth (scorecard, per-concern gaps, priority +order) — it is the standard new work gets evaluated against, not just a +backlog item; read it before framing anything and cite the section a +framing doc serves. The active-work ledger carries volatile branches, checkpoints, verification, and exact cross-machine recovery commands. -Keep both updated according to their own update protocols. +Keep all three updated according to their own update protocols. Always true, independent of the handoff: @@ -14,7 +18,11 @@ Always true, independent of the handoff: (`pmacs-protocol`). `#![forbid(unsafe_code)]`. - Workflow: framing doc in `docs/` -> user approval -> branch -> implement -> full gate suite -> PR -> user review rounds -> user says when to - merge. Never merge unprompted. One feature, one branch, one PR. + merge. Never merge unprompted. One feature, one branch, one PR. A + framing doc for coherence-affecting work should state its coherence + impact (journey steps touched, interaction islands added, config + registry adoption, background-work attribution) per `COHERENCE.md` + §20. - Gates before any PR: `cargo fmt --check`; `cargo clippy --workspace --all-targets -- -D warnings` (as its own step); `cargo test --lib`; `cargo test --lib --features crdt`; the touched acceptance suites; diff --git a/CLAUDE.md b/CLAUDE.md index 58c28f9..823b6e5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,11 +1,15 @@ # pmacs agent instructions -**Start here: read `docs/agent-handoff.md`, then +**Start here: read `docs/agent-handoff.md`, then `COHERENCE.md`, then `docs/active-work.md`, before taking on any work.** The handoff carries durable project state, working method, substrate invariants, and the -standing backlog. The active-work ledger carries volatile branches, +standing backlog. `COHERENCE.md` carries the product-coherence thesis +and its audited ground truth (scorecard, per-concern gaps, priority +order) — it is the standard new work gets evaluated against, not just a +backlog item; read it before framing anything and cite the section a +framing doc serves. The active-work ledger carries volatile branches, checkpoints, verification, and exact cross-machine recovery commands. -Keep both updated according to their own update protocols. +Keep all three updated according to their own update protocols. Always true, independent of the handoff: @@ -14,7 +18,11 @@ Always true, independent of the handoff: (`pmacs-protocol`). `#![forbid(unsafe_code)]`. - Workflow: framing doc in `docs/` -> user approval -> branch -> implement -> full gate suite -> PR -> user review rounds -> user says when to - merge. Never merge unprompted. One feature, one branch, one PR. + merge. Never merge unprompted. One feature, one branch, one PR. A + framing doc for coherence-affecting work should state its coherence + impact (journey steps touched, interaction islands added, config + registry adoption, background-work attribution) per `COHERENCE.md` + §20. - Gates before any PR: `cargo fmt --check`; `cargo clippy --workspace --all-targets -- -D warnings` (as its own step); `cargo test --lib`; `cargo test --lib --features crdt`; the touched acceptance suites; diff --git a/COHERENCE.md b/COHERENCE.md new file mode 100644 index 0000000..1594b69 --- /dev/null +++ b/COHERENCE.md @@ -0,0 +1,1547 @@ +# Product Coherence for Pmacs + +## Status of this document + +This document has two jobs. It states the **product-coherence thesis** +for pmacs, and it records the **audited ground truth** of how the +codebase measures against that thesis, so that no future agent or +contributor has to re-excavate it. + +- The vision prose is durable. The **Ground truth** subsections were + established **2026-07-25** by a four-lane code audit (discoverability, + interaction islands, packages/workers, first-run journey) plus a + distribution check, on branch `lsp-multi-root-affinity` + (= `main` @ `0827dd1` plus the multi-root LSP work). +- Citations name **symbols first, `file:line` second**. Line numbers + drift with the tree — `docs/keybindings.md` drifted by 250–1000 lines + within days of its "last verified" stamp (§24) — so treat the symbol + name and the structural claim as authoritative and the line number as + a hint. Re-grep before relying on a number. +- Grades used below: **Strong / Partial / Weak / Missing** (and + **Broken** where something actively fails). +- Update protocol is §25. When a PR changes any audited claim here, + updating this file rides that PR, the same way `docs/agent-handoff.md` + does. + +Relationship to the other required documents: `docs/agent-handoff.md` +carries durable project state and working method; `docs/active-work.md` +carries volatile branches and recovery; `docs/side-quest-backlog.md` +carries item-level deferrals. This document carries **product direction +and the measured distance to it**. It is not a second backlog; it is the +standard the backlog gets ranked against. + +--- + +## Purpose + +Pmacs already has an unusually strong technical foundation for an editor +at its stage of development. Its daemon/frontend split, semantic +rendering protocol, CRDT-based editing, structured worker runtime, Lua +programmability, language tooling, package resolver, terminal support, +and remote-capable architecture all point toward a system with genuine +long-term differentiation. + +The next challenge is not primarily adding more isolated capabilities. +It is making the existing and planned capabilities converge into a +coherent product. + +Visual Studio Code is used throughout this document as a reference point +because it is an exceptionally successful modern editor. Pmacs is +obviously not trying to become VS Code. Its goals are substantially +different: live programmability, inspectability, stronger concurrency +semantics, frontend plurality, and deeper user control are central to +Pmacs in ways they are not central to VS Code. The useful lesson is +therefore not to copy VS Code's interface or architecture wholesale, but +to understand how a technically complex system can become immediately +useful, progressively discoverable, and easy to adopt. + +The deeper reference point is Emacs. Emacs's beauty comes from its +ontological unity: the editor is text, Lisp, commands, buffers, and a +running system that the user can interrogate and change. Its enduring +achievement is not any single feature, but that it created the kind of +environment in which generations of users could build almost anything. + +Pmacs should preserve that unity while correcting the accidental +historical constraints beneath it: cooperative rather than general +parallelism, unclear ownership, global mutation, difficult unloading, +rendering coupled too closely to the core, opaque latency, implicit +remote context, and inconsistent package lifecycle. + +Pmacs does not need to contain everything Emacs contains before it can +be considered a successor. It must instead remain the kind of system in +which everything Emacs contains could eventually be built — with clearer +ownership, stronger concurrency, richer frontends, explicit execution +locations, and fewer historical traps. + +That places the VS Code comparison in its proper role. VS Code +demonstrates how a complex development environment can be coherent, +approachable, and immediately useful. Emacs demonstrates how an editor +can become a live, fertile, user-transformable world. Pmacs should +combine the adoption discipline of the former with the programmability +and unity of the latter. + +The core product objective should be: + +> **Pmacs should be immediately excellent, progressively understandable, +> completely inspectable, and ultimately replaceable.** + +A user should receive a polished workstation before they become an +editor engineer. If they choose to become one, the entire system should +remain open to them. + +--- + +## 0. Scorecard (audited 2026-07-25) + +| § | Concern | Grade | One-line state | +|---|---|---|---| +| 2 | Golden product journey | **Broken at entry** | `pmacs .` exits 1; only "launch" and "edit" pass cleanly zero-config | +| 3 | Zero-configuration state | **Partial** | Defaults genuinely strong; missing-tool failure is silent, not graceful | +| 4 | Progressive disclosure | **Inverted** | The advanced level is real; the beginner level is the missing one | +| 5 | Unified discoverability | **Substrate without surface** | Best-in-class registration metadata; almost no way for a user to reach it | +| 6 | Interaction islands | **Weak, and growing** | Six hardcoded key-interception shadows; no transient-keymap mechanism exists | +| 7 | First-class workspaces | **Missing (conventions only)** | Marker walk + four independent consumers; no workspace object | +| 8 | Execution locations | **Missing (architecture ready)** | SSH attach works; "location" is not a value anywhere | +| 9 | Worker ownership | **Mechanism without identity** | Cancellation solid; no owner/purpose/hierarchy; four disjoint activity views | +| 10 | Extension trust classes | **Missing (one class)** | Shared Lua state, `__index = _G`; MCP is the one out-of-process seam | +| 11 | Config layering + provenance | **Partial (foundation only)** | Typed registry is right; 5 settings live in it; no value provenance | +| 12 | Profiles | **Missing** | One hardcoded default keymap; not a named concept | +| 13 | Package lifecycle UX | **Resolution without lifecycle** | Mature resolver/lockfile; init-only install; no uninstall/disable/search | +| 14 | Workbench primitives | **Partial (best trajectory)** | Listview is a real shared primitive; bottom panel landed (#155) | +| 15 | Contextual affordances | **Weak** | Right-click menu only; code actions apply first-blindly; no git integration at all | +| 16 | Semantic frontend | **Strong** | v6..=v20 negotiated protocol; degradation practiced; TUI/GPU share the model | +| 17 | Distribution | **Missing** | CI is test-only; no binaries, channels, checksums, or update path | +| 18 | Onboarding | **Missing** | No welcome, no tutorial; `C-h` deletes a word; `M-x` is the only door in | +| 19 | Coherence acceptance tests | **Missing (culture ready)** | Superb per-arc acceptance discipline; zero cross-subsystem journey tests | + +Three cross-cutting patterns explain most of the table; they are +detailed in §1.1–§1.3: **substrate without surface**, **the silence +asymmetry**, and **per-arc coherence debt**. + +Coherence-shaped work already in flight at audit time: find-file / +dired Stage 0 (`C-x C-f`, PR #162, `docs/dired-framing.md`), bottom +panel Stage 1 (merged #155), multi-root LSP affinity (branch +`lsp-multi-root-affinity`), the config registry foundation (merged +#127). + +--- + +## 1. The Product Problem + +Pmacs is building many difficult things correctly and in parallel. That +is appropriate for an early systems project. The risk is that the +project succeeds architecturally while remaining fragmented +experientially. + +A technically sophisticated editor can still feel incoherent when: + +- installation requires repository knowledge; +- capabilities exist but are difficult to discover; +- subsystems expose unrelated interaction conventions; +- project, process, terminal, language-server, and remote state are + modeled separately; +- configuration is powerful but provenance is unclear; +- packages can extend the editor but cannot be understood, controlled, + or attributed; +- background work is concurrent but not meaningfully owned; +- new users must configure the system before they can experience its + strengths. + +The relevant distinction is between **capability completeness** and +**product coherence**. Capability completeness asks "can pmacs do X?". +Product coherence asks whether a user naturally encounters X at the +right time, whether X behaves by shared conventions, whether the user +can understand why X is active, and whether X feels like part of one +editor rather than an adjacent demonstration. + +Pmacs is well on its way toward capability completeness in several major +areas. Product coherence must now become an explicit development track +rather than an emergent consequence of subsystem work. The 2026-07-25 +audit found that every one of the eight bullet points above is true of +pmacs today, and that they share three structural causes. + +### 1.1 Ground truth: substrate without surface + +The single most consistent audit finding, appearing independently in all +four lanes: **the mechanism layer is disciplined, often best-in-class; +the product surface that would make it perceptible is missing.** The +July 2026 roadmap named an instance of this "dark matter — built but +unwired" and treated it as a one-time backlog. It is not one-time; it is +the project's default failure mode. The audited inventory of complete, +working, unreachable capability: + +- **The entire rich help system.** `src/help.rs` implements a + self-navigable `*help*` buffer with `[command:]` / `[key:]` / + `[mode:]` / `[hook:]` / `[buffer:]` / `[view:]` cross-reference links + and `follow_link_at`, installed as `pmacs.help.show_command` / + `show_key` / `show_buffer` / `show_mode` / `show_hook` / `show_view` + (`install_help_module`, `src/lua_bindings/mod.rs:5597`). `grep -rn + "pmacs.help" builtin/` returns **zero hits** — no command, no + keybinding, no caller. +- **File-name completion.** `CompletionSource::Files { root }` + (`src/minibuffer.rs:589`) is reachable from Lua as `source = "files"` + + `source_root` — zero builtin callers. +- **Command availability.** `Command.predicate` is stored on every + command and **never evaluated** by `invoke`, `invoke_interactive`, + keymap dispatch, M-x filtering, or the menu (§5). +- **Package ownership.** `CurrentlyLoadingPackage` is a stack correctly + pushed/popped around every package chunk + (`src/lua_bindings/mod.rs:3953-3966`) and consulted by exactly one + binding (`on_unload`'s fallback). Every registrar ignores it (§13). +- **LSP health.** `LspManager::status_buffer_text()` (`src/lsp.rs:1204`) + is Lua-bound; no builtin command opens `*lsp*` (§2, §9). +- **Interactive file opening.** `pmacs.buffer.find_or_open` + (`src/lua_bindings/mod.rs:3103`) had no interactive caller at audit + time; a complete 1,384-line dired exists as a frozen test fixture + (`tests/fixtures/pmacs-dired/init.lua`). Being fixed now: dired Stage + 0 (PR #162). + +The strategic consequence: **most coherence gaps in pmacs are doors, +not engines** — deliberately deferred surface, not design error. That is +the cheap kind of gap, and it should change how the remaining work is +costed. + +### 1.2 Ground truth: the silence asymmetry + +Synchronous, user-initiated failures report well: `M-x` errors surface +as `"M-x error: "` (`builtin/commands/default.lua:633-641`), +compile spawn failures print in-buffer and on the status line +(`builtin/runtime/compile.lua:850-855`), and `pmacs --gpu` with no +`pmacs-gpu` binary produces the best missing-tool message in the +codebase — it names both the sibling path it tried and the PATH fallback +(`src/main.rs:367-379`). + +Automatic, background failures are swallowed. The canonical case, hit on +**every file open** when a language server is preconfigured but not +installed: `Command::spawn` ENOENT propagates up through +`LspManager::spawn` and raises in Lua — where `ensure_server` `pcall`s +it and returns nil (`builtin/runtime/lsp.lua:614-626`), and the +`buffer.after-load` hook `pcall`s the whole attach +(`builtin/runtime/lsp.lua:895-897`). Net user-visible result: nothing. +No status message, no `*errors*` entry, no modeline marker (the LSP +segment is gated on an attachment record existing, so absence is +indistinguishable from "unsupported file type"). Working tree-sitter +highlighting **actively masks** the failure — the user sees colored text +and assumes language intelligence is on. Post-crash is the same shape: +`LspEventKind::Crashed` is pushed (`src/lsp.rs:2394`) and no builtin +subscriber surfaces it. + +This directly contradicts the product thesis (§23): the "without +freezing" half is delivered; the "without becoming opaque" half is +currently false for exactly the failures a new user will hit first. + +**Rule to adopt:** anything that fails automatically must leave a +user-visible trace with a named owner. A `pcall` around background +wiring must log attributed failure, never discard it. + +### 1.3 Ground truth: coherence debt compounds per-arc + +Three audited growth patterns show subsystem work accruing coherence +debt with no counter-pressure: + +- Each new modal UI **extended the shadow family** instead of building + the keymap-layer mechanism (menu → completion → query-replace, §6) — + and each addition must hand-sync three guard lists (`dispatch_key`, + `dispatch_idle_for`, `dispatch_paste`). +- Each new subsystem **added its own activity view** (`*workers*`, + `pmacs.process.list`, `*lsp*`, the terminal-private id set, §9), + because no common identity key exists to join them. +- Each new option **individually decides** whether to adopt the config + registry; five have, everything else has not (§11). + +The framing-doc workflow (scout → framing → approval → acceptance +criteria → bite-verified review) is exactly the right tool to reverse +this — no framing has ever carried a product-coherence acceptance +criterion. Adding them is a process change, not an engineering arc, and +it is what makes this document *required* rather than advisory. + +--- + +## 2. The Golden Product Journey + +Pmacs should maintain one protected end-to-end experience against which +all major work is tested: + +1. Install Pmacs. +2. Launch it without prior configuration. +3. Open a real project. +4. Understand the visible interface. +5. Edit immediately. +6. Receive language intelligence. +7. Find a symbol or file. +8. Open a terminal. +9. Build or test the project. +10. Inspect and act on an error. +11. Understand what background work is running. +12. Close and later restore the workspace. + +This does not need to exercise every advanced feature. It exists to +prove that the editor's components form a usable whole. A strong initial +target is a Rust project, because Rust stresses many of pmacs's intended +strengths: project detection, toolchain discovery, language-server +lifecycle, async diagnostics, build/test integration, terminal use, +large compilation workloads, symbol search, background indexing, +structured error presentation. + +```text +Install Pmacs + ↓ +Run `pmacs .` + ↓ +Project root detected + ↓ +Rust mode activated + ↓ +rust-analyzer found or installation guidance shown + ↓ +Files, diagnostics, terminal, and project actions available + ↓ +Build or test command discoverable + ↓ +Errors become navigable structured results +``` + +This journey should become a release gate. New architectural work should +be evaluated partly by whether it improves, preserves, or complicates +the journey. + +### Ground truth: the journey today + +**Grade: broken at step 3.** Verified empirically at audit time: + +``` +$ ./target/release/pmacs . +pmacs: Is a directory (os error 21) +EXIT=1 +``` + +The literal first arrow of the diagram above fails. `load_file` +(`src/file_io.rs:81-87`) does `File::open` (succeeds on a directory) +then `read_to_end` → EISDIR, which is not `NotFound`, so +`EditorState::open` returns `Err` and `main` prints and exits +(`src/main.rs:411-414`). Multiple file arguments are also rejected +(`"multiple files not yet supported"`, `src/main.rs:227`). Everything +from step 6 onward is gated on a file being open, and the only +zero-config way to open one is naming it on the command line — which +requires already knowing the path. + +Full verdict table: + +| # | Step | Verdict | Evidence | +|---|---|---|---| +| 1 | Install | **Partial** | Source build only: `cargo build --release --workspace --features pmacs/crdt` (`README.md`). No binaries, no packaging. Runtime deps (`/bin/sh`, git, tar, coreutils) documented, never checked at runtime | +| 2 | Launch unconfigured | **Works** | `EditorState::new()` → empty `*scratch*`; missing config is not an error (`src/config.rs:7-9`); recentf/saveplace/autosave default-on | +| 3 | Open real project | **Missing** | `pmacs .` exits 1 (above). No directory handling anywhere | +| 4 | Understand interface | **Partial** | Mode line gives name/modified/L:C/scroll + mode/LSP/terminal segments; but no welcome text (`EditorCore::new` sets `status: String::new()`), no cheat sheet, and `C-h` deletes a word (§18) | +| 5 | Edit | **Works** | Full CUA + Emacs keymap in 161 lines (`builtin/keymaps/default.lua`); isearch, query-replace, kill ring, undo/redo, auto-indent/pair/comment, atomic save. Genuinely excellent zero-config | +| 6 | Language intelligence | **Partial** | Rust grammar bundled and auto-attaches; rust-analyzer preconfigured (`builtin/runtime/lsp.lua:44-52`) — but a missing binary fails silently (§1.2) and highlighting masks it. No LSP status command exists to diagnose | +| 7 | Find symbol / file | **File: missing → in flight (PR #162). Symbol: works but undiscoverable** | No find-file/dired/picker existed at audit; `M-.`/`M-?`/`C-c o` bound but advertised nowhere and server-gated; no workspace-symbol command; `pmacs.index.*` has no UI | +| 8 | Open terminal | **Works but undiscoverable** | Full PTY with scrollback + modeline segment — reachable only as `M-x terminal`, no keybinding | +| 9 | Build / test | **Partial** | `M-x compile.run` works, defaults cwd to detected project root, parses Rust `-->` errors — but no keybinding, an **empty first prompt** (`initial = last and last.cmdline or ""`, `builtin/runtime/compile.lua:1134-1138`), and no `cargo build`/`cargo test` suggestion despite `ProjectKind::Cargo` existing (`src/project.rs:77`) | +| 10 | Inspect error | **Partial (good once reached)** | `E:n W:n` modeline counts, underlines, `M-g n/p` + ``C-x ` `` walking a unified compile/grep/diag source, message echo, `RET` visits. Gated entirely on step 6 or 9 succeeding first | +| 11 | See background work | **Works but undiscoverable** | `*workers*` view via `M-x editor.list-workers`; `C-c C-k` cancel-at-point. No keybinding, no statusline spinner/progress indicator anywhere (§9) | +| 12 | Close + restore | **Partial** | Per-file cursor+scroll (saveplace), recent files, minibuffer history, autosave recovery all restore zero-config. Open-buffer set and window layout do **not**: desktop-save is opt-in (`pmacs.session.desktop_mode(true)`) *and* a documented no-op under a daemon (`src/desktop.rs:323-326`, `:353-356`, Q#DS9) | + +A journey observation worth keeping verbatim from the audit: +**keybinding coverage is inverted relative to frequency** — `C-c @ +C-M-s` opens all folds, while opening a file, opening a terminal, and +running a build have no bindings at all. + +--- + +## 3. A Strong Zero-Configuration State + +Pmacs should not require configuration before it becomes pleasant. The +default experience should demonstrate the editor's thesis: responsive +editing, visible asynchronous work, coherent project awareness, language +intelligence, integrated terminal and task execution, helpful +diagnostics, discoverable commands, graceful failure when external tools +are absent. + +Configuration should be an escalation path: + +1. The editor works. +2. The user notices a preference. +3. The relevant setting or command is easy to find. +4. The user changes it. +5. The editor explains where the effective value came from. +6. Advanced users can replace the behavior entirely. + +### Recommended default surface + +The graphical frontend should have a deliberate default workspace with a +restrained number of visible regions: main editor area; compact +statusline; optional project/files surface; bottom panel for terminal, +build output, diagnostics, and other transient tools; command palette; +contextual actions; unobtrusive background activity indicator. The TUI +should express the same conceptual model within terminal constraints. +The goal is not identical geometry across frontends — it is shared +nouns, commands, lifecycle, and state. + +### Ground truth + +**Grade: partial — the defaults half is strong, the graceful-failure +half fails.** + +What already works with zero configuration, and is a real asset: + +- Missing config is **not an error by contract** (`src/config.rs:7-9`); + no config directory is created or required; a *broken* `init.lua` + does not block startup — the error lands in `*errors*` and the status + line (`src/config.rs:10-13`). +- Default-on persistence: recentf (`builtin/runtime/recentf.lua`, cap + 50, `C-x C-r`), saveplace (`builtin/runtime/saveplace.lua`, restores + cursor + view on `after-load`), autosave every 30 s with next-session + recovery (`builtin/runtime/autosave.lua:24`), per-bucket minibuffer + history. State root: `PMACS_STATE_HOME` → `$XDG_STATE_HOME/pmacs` → + `~/.local/state/pmacs` (`user_state_dir`, `src/state.rs:54-70`), + wired only in real entry points (`install_state_dirs`) so tests stay + hermetic. +- Atomic saves, full editing surface, bundled grammars for every + preconfigured LSP language. + +What fails the escalation path: + +- Step 3 ("easy to find") fails for both settings and commands (§5). +- Step 5 ("explains where the value came from") is **unanswerable + today**: config overrides are stored as bare values with no source + (§11). +- "Graceful failure when external tools are absent" is the silence + asymmetry (§1.2). The `--gpu` message (`src/main.rs:367-379`) is the + pattern to replicate; LSP auto-attach is the anti-pattern. + +--- + +## 4. Progressive Disclosure + +Pmacs should support several levels of use without requiring users to +inhabit the most advanced one. These levels should be different +presentations of the same underlying objects — a command selected from a +context menu, invoked through `M-x`, bound to a key, called from Lua, or +triggered by an agent should be the same command object. + +### Ground truth + +**Grade: inverted.** The advanced level is largely real; the beginner +level is the one missing. Audited level-by-level: + +**Beginner** (should see: files, buffers, search, diagnostics, terminal, +build actions, menus, missing-tool guidance): + +- files ✗ (no find-file at audit; PR #162 in flight) · buffers ✓ (`C-x + b`, `*buffer-list*`) · search ✓ (`C-s`/`C-r`/`C-M-s`; project.search + is M-x-only) · diagnostics ✓ once a server runs · terminal ✓ but + M-x-only · build ✓ but M-x-only with empty prompt · menus △ + (right-click only, 11 items) · missing-tool guidance ✗ (§1.2). + +**Intermediate** (should discover: palette, keybinding search, workspace +settings, profiles, package management, task definitions, +frontend/language settings): + +- palette △ (`M-x` fuzzy over bare names, §5) · keybinding search ✗ (no + list-keybindings/where-is commands) · workspace settings ✗ (no + workspace scope, §11) · profiles ✗ (§12) · package management ✗ + in-session (§13) · task definitions ✗ · frontend customization △ + (themes, `pmacs.gpu.set_font`, statusline providers — all Lua-only) · + language settings △ (raw Lua tables, outside the registry). + +**Advanced** (should be able to: inspect implementations, redefine live, +create packages, new views, providers, keymap layers, workspace policy, +orchestrate workers, replace interaction models): + +- inspect ✓ (SourceLocation on everything; no jump-to-source command + though) · redefine live ✓ (`unregister` + `define`) · packages ✓ + (authoring is real, §13) · new views ✓ (listview is Lua-usable) · + providers ✓ (statusline; completion/minibuffer sources are a fixed + Rust vocabulary) · keymap layers ✗ (§6 — the mechanism does not + exist) · workspace policy ✗ · orchestrate workers △ + (`pmacs.workers.register` funnels into builtin dispatchers, §9) · + replace interaction models ✗ (the shadows, §6). + +The "same command object" principle largely holds where surfaces exist — +menu items, keybindings, and M-x all resolve command names into the one +registry — with one caveat: menu items are a **parallel registry of +labels** whose command references are unvalidated (§5). + +--- + +## 5. Unify Discoverability + +Pmacs already has the beginnings of a strong command registry. This +should become the center of a broader discovery model. Every meaningful +action should eventually expose: stable symbolic identity, title, +description, category, aliases, current keybindings, provenance, +applicability predicate (with an explanation when unavailable), argument +schema, destructive/asynchronous/reversible flags, locality, related +commands and settings, and source location. Settings should expose name, +type, description, default, effective value, provenance, scope, +validation rules, listeners, related commands. Packages and workers +should expose the analogous sets (§13, §9). + +This suggests a general pmacs principle: + +> **Anything that can affect the user should be discoverable as a +> structured object with identity, provenance, ownership, and +> lifecycle.** + +### Ground truth + +**Grade: substrate without surface — the sharpest instance of §1.1.** + +**What the substrate already has (genuinely strong):** + +- `Command` (`src/command.rs:66-79`) = `{ name, description, source, + body, predicate }`. Description is **mandatory and validated** (R42); + duplicate names are a hard error, not an overwrite; `SourceLocation + { file, line }` is auto-captured from Lua debug info on **every** + command, hook, menu item, config definition, config listener, and + keybinding — the user cannot forge it. ~147 `pmacs.command.define` + sites across `builtin/`. +- `ConfigDefinition` (`src/config_registry.rs:396-410`) is **richer + than `Command`**: name, mandatory description, `ConfigKind` + (Boolean/Integer/Number/String/Enum with bounds, choices, + allow_empty), default, `Live`/`StartupOnly` mutability, source. + `pmacs.config.list()` returns full descriptor tables. +- Reverse keybinding lookup exists as data: `KeymapStack::iter_all()` + (`src/keymap_stack.rs:295-311`) enumerates every binding; + `pmacs.describe.command(name).key_bindings` computes where-is on + demand. +- `pmacs.describe.*` (`src/lua_bindings/mod.rs:6042-6162`) returns + structured tables for command/key/buffer/view/mode/hook, and + `describe.key` resolves against the **active buffer + major mode**. +- M-x matching is fuzzy (case-insensitive subsequence with + boundary/consecutive bonuses, `fuzzy_score`, + `src/minibuffer.rs:637-666`). + +**What is missing, itemized:** + +- `Command` has **no title, no category, no aliases, no argument + schema, no destructive/async/reversible flags**. The dotted-name + prefix (`buffer.`, `lsp.`) is convention, not data. MCP tooling works + around the missing schema by stuffing rendered JSON schema text into + the description string. +- **`Command.predicate` is dead metadata.** It is read in exactly two + places (a literal line in the unreachable help renderer, and a test) + and **never evaluated** by `invoke`, `invoke_interactive`, dispatch, + M-x filtering, or the menu. The doc comment's claim that "the command + palette (T M2.7) uses it to gray out unavailable entries" describes + something that never shipped. +- **M-x shows bare name strings.** `CompletionSource::Commands` returns + `Vec` of names; the wire type `MinibufferPrompt.candidates` + is `Vec` (`pmacs-protocol/src/message.rs:994-1006`). No + description, no keybinding, no category alongside candidates — while + `CompletionPopupRow` (`:1231`) already carries `kind` and `detail`, + proving richer rows are a solved wire problem in this codebase. +- **The entire Rust help layer is orphaned** (§1.1). Consequence: two + parallel `*help*` implementations exist — `help.rs`'s + cross-referenced renderer and the Lua `show_help_text` in + `builtin/commands/default.lua:1103-1136` — and the one users can + actually reach (`M-x editor.describe-command`) renders **less** than + the unreachable one (no source, no scope, no predicate note). +- **Missing as commands entirely:** describe-key, describe-mode, + describe-hook, describe-buffer, where-is, list-commands, + list-settings, list-keybindings, apropos. What exists: + `editor.describe-command`, `editor.describe-setting`, + `editor.describe-instance[-buffer]`, `editor.list-buffers`, + `editor.list-workers`. `M-x describe-setting` prompts **free-text + with no completion source** (deliberately skipped — + `builtin/commands/default.lua:1180-1185`); a typo yields a status + line error. +- **No help prefix key.** `C-h` is `buffer.delete-word-backward` + (`builtin/keymaps/default.lua:86`, with a comment noting the key "was + free"). No `F1`, no `C-h k/f/b`. +- **Settings value provenance is absent.** Overrides are stored as bare + values (`global: HashMap`, + `src/config_registry.rs:693-708`); `describe-setting`'s "Source:" is + the *definition* site. "Why is this setting 4 and who set it?" is + unanswerable (§11). +- **Menu items are a parallel registry.** `MenuItem` + (`src/menu.rs:56-78`) carries its own hand-written `label` duplicating + the command's description, with a lazily-resolved `command` name + string that is **never validated to exist** — a typo'd item silently + does nothing when clicked. The wire row is label + separator only + (`MenuPromptRow`): no key hints, no grayed state. Note the asymmetry: + `pmacs.menu.list` reports `has_predicate`; `pmacs.describe.command` + does not. +- **The two key-lookup APIs disagree.** `pmacs.keymap.lookup` is + global-only (it resolves with no buffer and no modes, + `src/lua_bindings/mod.rs:6294-6307`) while `pmacs.describe.key` is + context-aware. `pmacs.keymap.list` erases `source` and renders + `Scope::Buffer(id)` as bare `"buffer"` (id erased), so full-fidelity + enumeration requires per-command `describe.command` calls. There is no + which-key-style prefix surface. + +**Shape of the fix:** roughly (a) three metadata additions on `Command` +(title, category, predicate actually evaluated + reported), (b) value +provenance in the config registry, (c) a dozen interactive commands and +richer M-x candidate rows over introspection that **already exists**. +This is the highest payoff-per-effort concern in the document. + +--- + +## 6. Eliminate Hardcoded Interaction Islands + +Pmacs's public programmability story will be strongest when all major +interaction layers pass through ordinary registries and extension +points. Temporary or modal interfaces — incremental search, query +replace, minibuffer prompts, completion menus, context menus, transient +selectors — should eventually use inspectable keymap layers rather than +special Rust-level interception. A general transient keymap model +includes priority, activation condition, owner, lifetime, fallback +behavior, discoverability, help labels, and cancellation behavior. + +### Ground truth + +**Grade: weak, and growing by one island per modal feature.** + +Everything funnels through one function: `EditorInstance::dispatch_key` +(`src/editor.rs:901`), a single input-precedence state machine (its own +`#[allow(too_many_lines)]` says as much). The audited precedence order: + +| # | Surface | Guard site | Decoder | Kind | +|---|---|---|---|---| +| 0 | popup-vs-modal auto-close | `editor.rs:917-925` | — | pre-step | +| 1 | Context menu | `editor.rs:933` | `MenuKey::from_chord` (`editor.rs:3005`) | **full shadow** | +| 2 | isearch | `editor.rs:939` | `SearchKey::from_chord` (`editor.rs:2902`) | **full shadow** | +| 3 | query-replace | `editor.rs:945` | `QueryReplaceKey::from_chord` (`editor.rs:2967`) | **full shadow** | +| 4 | Minibuffer | `editor.rs:951` | `MinibufferAction::from_chord` (`src/minibuffer.rs:468`) | **full shadow** | +| 5 | Completion popup | `editor.rs:958-971` | `CompletionPopupKey::from_chord` (`editor.rs:3056`) | **partial shadow** (control chords only; skipped while a multi-key prefix is pending) | +| 6 | Terminal transport + `C-c` escape | `editor.rs:973-1010` | `is_terminal_escape_chord` (`editor.rs:4355`) | **partial, transport-level** | +| 7 | Ordinary dispatch | `editor.rs:1018-1032` | `KeymapStack::resolve` | the only inspectable layer | + +Facts that define the gap: + +- **Full shadows eat every key**, including unrecognized ones (each + decoder has an `Ignore`/`Dismiss` fallback arm). While a terminal + buffer is focused and unescaped, *all* keys encode to the child — + `C-c`-leading user bindings are **structurally unreachable** in a + terminal buffer. +- **No transient-keymap mechanism exists to migrate to.** `KeymapStack` + has exactly three fixed scopes — `Buffer(BufferId)`, `Mode(String)`, + `Global` (`src/keymap_stack.rs:37-44`); resolution order buffer → + mode → global with cooperative prefix-pending across scopes + (`resolve`, `keymap_stack.rs:235-291`). No layer stack, no push/pop, + no priority, no lifetime. The Lua scope accept-list hard-rejects + anything else. So this is not "migrate the shadows to the layer + system" — **the layer system must be built first.** (`active_modes` + is also at most one mode today; minor modes are unbuilt.) +- **`describe-key` lies while a shadow is active.** With the completion + popup open, `describe-key C-n` reports `cursor.down @global`; the + literal arm `'n' => Some(Self::Next)` fires instead. Introspection + has zero awareness of the shadows; Lua can observe only a boolean per + surface (`popup_visible`, `search_active`, `query_replace_active`, + minibuffer-active). +- **This is deliberate and documented** — rationale R51 + (`docs/keybindings.md`, `src/minibuffer.rs:470`): the shadows are + intentionally not user-configurable. The completion framing + considered and rejected buffer-local binds on teardown-lifecycle + grounds (`docs/in-buffer-completion-framing.md:93-105`) — the + objection was a *leaked binding outliving its session*, which is an + argument for a lifetime-owning layer handle, not against layers. +- **Three hand-synced guard lists** must be updated per shadow: + `dispatch_key`, `dispatch_idle_for` (`editor.rs:791` — deliberately + omits the partial popup shadow; load-bearing for CRDT frontends' + optimistic-apply correctness), and `dispatch_paste` + (`editor.rs:1129-1140`). +- Off-path hardcodes: client-side **F12 detach** (`is_detach_key`, + `src/attach.rs:997-1006`) and the GPU **optimistic key classifier** + (`crate::optimistic::classify_key`) — the latter is classification, + not routing, and is kept honest by `dispatch_idle_for`. + +**The counter-example that proves the idiom:** the entire picker/panel +family — listview (references, outline), project-search, buffer-list, +compile-mode, REPL, terminal scroll commands — uses ordinary +**buffer-local keymaps** via `pmacs.keymap.bind { scope = "buffer" }` +(`builtin/runtime/listview.lua:76-88` and siblings). These are +inspectable, correctly reported by describe-key, and rebindable from +`init.lua`. Roughly half the transient UI already lives on the right +side of the line. + +**The concrete missing primitive** is small and well-scoped: a transient +overlay consulted before buffer scope (a `Scope::Transient` or an +overlay `Vec`), with (a) push/pop tied to session lifetime via a +lifetime-owning handle (RAII on the Rust side), (b) a full-shadow vs +partial-shadow flag (isearch eats everything and falls back to +search-self-insert; the popup intercepts eight chords and falls +through), and (c) `dispatch_idle_for` **derived** from the stack ("any +active layer is full-shadow") instead of hand-maintained. With that, the +six ladder rungs collapse into "session pushes a layer on open, pops on +close," and describe-key becomes truthful for free. + +--- + +## 7. First-Class Workspaces + +Project-root detection is useful, but pmacs needs a richer workspace +object. A project answers "which root contains this file?"; a workspace +answers "which persistent development environment owns this set of +activity?" A workspace should eventually own: + +```text +Workspace +├── identity +├── one or more roots +├── execution location +├── environment and toolchain +├── configuration layers +├── trust policy +├── enabled packages +├── language-server instances +├── indexes +├── terminals and processes +├── tasks +├── debugger sessions +├── open buffers and views +├── frontend layout state +└── persistence and restoration policy +``` + +This matters for multi-root language servers, monorepos, generated +files, remote projects, containers, HPC environments, per-project +packages, task ownership, session restoration, and project-specific +trust. The workspace should be a core runtime entity, not an informal +convention shared across unrelated subsystems. + +### Ground truth + +**Grade: missing — what exists is a marker walk plus four independent +per-subsystem conventions.** + +- **Detection**: `src/project.rs` — `default_markers()` is + `Cargo.toml`, `go.mod`, `package.json`, `.git` (directory), with the + deliberate rule that **language markers outrank `.git`** at the same + ancestor level; upward walk, innermost wins; `set_search_boundary` + honored; `ProjectKind` (e.g. `Cargo`) exists and is **consumed by + nothing** user-facing. +- **Four independent consumers**, each resolving on its own: + LSP root (`project_root_for`, `builtin/runtime/lsp.lua:554-570`: + config override → marker walk → file's own directory, returning + `root, source` where source ∈ config/detected/fallback); compile cwd + (`project_root_of_active`, `builtin/runtime/compile.lua:600-608`); + project-search root (`resolve_search_root`, + `builtin/commands/default.lua:843-857`, falls back to `"."`); the + project symbol index (`.pmacs/index.json`, `src/project_index.rs`). +- **There is no "current project" independent of the active buffer's + path.** With only `*scratch*` open, every consumer above returns + nil/`"."`. Nothing owns the set {roots, servers, terminals, tasks, + layout} — which is why desktop-save under a daemon had nothing + principled to attach to (Q#DS9, §2 step 12). +- **First slice in flight**: the multi-root LSP server-affinity work + (branch `lsp-multi-root-affinity`) makes *(language, found-root)* the + server identity — the first time a root functions as an identity key + rather than a spawn parameter. Note it is again per-subsystem: LSP + learns roots; compile, search, index, and trust do not share the + object. + +A workspace entity is a **model gap** (real arc), not wiring. It is also +the prerequisite that keeps §8 (locations), §9 (task ownership), §11 +(workspace config scope), and step 12 of the journey from each inventing +their own ownership story. + +--- + +## 8. First-Class Execution Locations + +Pmacs's daemon/frontend architecture gives it an excellent basis for +remote development. The next step is to model execution location +explicitly — a value that can be inspected and assigned, not an +implementation detail hidden inside file access or process spawning: + +```text +Location +├── local +├── ssh://host +├── container://name +├── slurm://allocation +├── daemon://session +└── custom provider +``` + +Filesystem roots, processes, terminals, language servers, workers, +debuggers, indexers, package services, and build/test tasks should all +carry a location. That makes answerable: where is this server running? +where will this build execute? is this terminal local? can this worker +migrate? what happens if the remote daemon disconnects? + +### Ground truth + +**Grade: missing as a model; the architecture half already works.** + +What exists: `pmacs --attach user@host` (remote TUI over SSH), +`ssh:user@host/instance` / `local:/path.sock` addressing, mosh-modeled +reconnect-on-drop, and the daemon/frontend split itself — i.e. +`daemon://session` exists implicitly and robustly. What does not exist: +any `Location` value. Every `ProcessSpec` spawn, LSP server, terminal +PTY, and worker is implicitly daemon-local; no resource carries a +location field; nothing can be asked "where is this running?". No +container/slurm/provider concept anywhere. + +This concern is deliberately *after* §7 in dependency order: a location +without a workspace to scope it has nothing to attach to. For the +research/HPC ambition (§12's Research Workstation profile), this pair is +the long-lead differentiator — nothing else in the editor market models +it well. + +--- + +## 9. Extend the Worker Model into Structured Concurrency + +Pmacs's worker system is one of its most distinctive strengths. +Cancellation, supersession, streaming, frame-aware draining, and the +`*workers*` view provide a strong basis. The next step is ownership and +hierarchy: every substantial task should have an owner, a workspace, an +optional buffer/view, a parent, children, a latency class, a +cancellation scope, a resource budget, an execution location, progress, +and failure attribution. + +```text +Workspace: pmacs +└── Command: project-build + ├── Task: save-dirty-buffers + ├── Task: cargo-check + │ ├── Process: cargo + │ └── Stream: compiler-diagnostics + └── Task: refresh-diagnostics +``` + +Cancelling `project-build` should cancel its children. Closing a +workspace should terminate or detach workspace-owned work. Reloading a +package should stop package-owned tasks. The activity view should answer +what is running, why, who owns it, where, what depends on it, and what +cancellation will affect. That turns parallelism into a product feature +rather than an implementation claim. + +### Ground truth + +**Grade: mechanism without identity.** + +**The mechanism layer is solid:** cooperative per-job cancellation +tokens with panic isolation (`src/worker.rs:13-28`); supersession with +correct settle-time pruning (`src/async_runtime.rs:688-701`) — a +genuinely good primitive; a completions ring (cap 64); `register_external` +so non-pool work (LSP requests, MCP) appears uniformly; one shared +`ProcessSupervisor` under everything (`src/editor.rs:341`); the +`*workers*` view (`src/workers_buffer.rs`, opened by `M-x +editor.list-workers`, auto-refreshing, `C-c C-k` cancel-at-point). + +**The identity layer is absent:** + +- `PendingJob` (`src/async_runtime.rs:365-392`) carries `{cancel, + state, supersede_key, stream_buffer, max_batch, kind, + dispatched_at}`. **No owner. No purpose string. No + workspace/buffer association. No parent.** The one buffer link that + exists (parse job → buffer) lives in a `SyntaxCoordinator` side map, + invisible to the workers view. +- `JobKind` is a **closed 12-variant enum** (Sleep, ComputeSum, EmitN, + Grep, Parse, FsReadDir, FsStat, FsRename, FsChmod, FsRemove, + McpRequest, LspRequest). `pmacs.workers.register` funnels Lua jobs + into existing Rust dispatchers, so **every third-party job renders + under a builtin's label**. +- Supersession is opt-in per dispatch site and underused: `"search"` + (grep) and `lsp:{method}:{sid}:{uri}` use it; **parse jobs and all + MCP requests pass `None`** — a fast typist stacks parse jobs. +- Cancellation scopes: per-id and per-key only. No cancel-all, + by-kind, by-buffer, by-owner, or by-subtree — there is no scope to + range over. +- **Four disjoint activity planes with no join key:** + +| Plane | Surface | What it misses | +|---|---|---| +| Async jobs | `*workers*` | processes, servers, terminals | +| OS processes | `pmacs.process.list` (no buffer view exists) | **filters to `LineOriented` only — terminal PTYs are invisible**; `spawn_terminal` bypasses the public path entirely | +| LSP servers | `*lsp*` status text | **no builtin command opens it**; LSP sets `RestartPolicy::Never` on the supervisor and runs its own restart logic | +| Terminals | private id set drained after the supervisor tick | user-visible in none of the above | + + A terminal PTY appears in **no** user-visible activity view. An LSP + server appears in `*lsp*` (unreachable) and `list()`; its requests + appear in `*workers*`; nothing joins them. +- **No progress indicator exists anywhere** — no statusline spinner, + no busy count (grep for progress/spinner/busy in `src/statusline.rs` + is empty). "Visible asynchronous work" (§3) is currently false unless + the user knows to run `M-x editor.list-workers`. +- `ProcessSpec.label` is the nearest thing to attribution: caller- + supplied, unvalidated convention (`lsp:{name}`, terminal buffer + name). + +The audit's conclusion, worth preserving verbatim: *because identity is +missing, scoped cancellation has nothing to scope over and a unified +activity view has nothing to group by — the four views exist precisely +because there is no common key to merge them on.* Owner/purpose/parent +fields on the job and process specs are the prerequisite; the unified +view and the ownership tree fall out of them. + +--- + +## 10. Define Extension Trust and Isolation Classes + +Pmacs should preserve live, low-friction programmability — it should not +force all extensions into rigid out-of-process APIs. At the same time, +namespace isolation inside a shared Lua state is not enough for fault +containment, security, latency containment, memory accounting, +native-code isolation, reliable unloading, or project-local trust. Pmacs +should define extension classes before the ecosystem becomes large: + +- **10.1 Trusted core packages** — in-process, deep API access, + distributed with pmacs or explicitly trusted. +- **10.2 Normal Lua packages** — shared/managed runtime, declared + capabilities, owned registrations and workers, execution budgets, + measurable latency, reloadable lifecycle, package-level error + attribution. +- **10.3 Isolated service extensions** — separate process, typed RPC, + crash recovery, resource accounting, explicit fs/process/network + permissions. +- **10.4 Project-local / untrusted** — explicit approval, restricted + capabilities, strong isolation, workspace-scoped trust, easy + revocation. + +### Ground truth + +**Grade: missing — one class exists.** + +Every package today is a 10.1/10.2 hybrid with none of 10.2's +machinery: in-process, per-package `_ENV` with `__index = _G` +(namespace hygiene, not containment), full API access, no capability +declarations, no budgets, no latency measurement, no owned-registration +lifecycle (§13). The only containment primitive in the tree is the +instruction-count hook that can cancel a hot-looping main-thread chunk +(`src/lua_isolation.rs:1-39`) — a runaway guard, not an isolation class. + +Two real assets to build on: the loader's `exports` gating (the package +searcher is deliberately inserted at position 1 of `package.searchers` +so exports are enforceable, `src/lua_bindings/mod.rs:3891-3900`), and +**MCP as the existing 10.3 seam** — packages can already spawn MCP +servers and consume their tools over a typed transport +(`docs/mcp-for-package-authors.md`), which is exactly the +separate-process/typed-RPC shape 10.3 asks for. Project-local trust +(10.4) has a natural anchor once §7's workspace exists. + +Sequencing note: 10.2's "owned registrations, reloadable lifecycle, +error attribution" is the same work as §13's ownership gap — do it once, +under one arc. + +--- + +## 11. Configuration as Typed, Layered Data + +Pmacs's typed configuration registry is the correct foundation. It +should develop into a layered system with explicit provenance. Likely +layers: built-in defaults; profile defaults; user settings; +machine-local; remote-location; workspace; root/folder; language/mode; +buffer-local; session overrides. A setting inspection view should show +the full chain and the active source: + +```text +setting: editor.tab-width +effective value: 4 +type: integer +scope: workspace + +defined by: + built-in default: 8 + Rust profile: 4 + user setting: 2 + workspace override: 4 + +active source: + ~/src/pmacs/.pmacs/settings.lua +``` + +Pmacs should also preserve three distinct levels — **settings** (typed +declarative data), **behavioral customization** (commands, hooks, +keymaps, Lua), **package construction** (new capabilities) — so that +users do not need executable Lua for ordinary preferences, while +advanced users can still replace the mechanism. + +### Ground truth + +**Grade: partial — the foundation shipped (#127) and is correct; the +layering, provenance, and adoption have not followed.** + +- The registry is typed, described, duplicate-rejected, freeze-aware + (`StartupOnly`), listener-bearing, and introspectable — see §5. Its + design decisions (always-store overrides, explicit buffer, no ambient + scope) are recorded in `docs/config-registry-framing.md`. +- **Two scopes exist** of the ten layers listed above: global and + buffer-local. Per-language and per-project are patterns (a hook + calling `set_local`), not scopes. No profile, workspace, machine, or + remote layer. +- **Value provenance is absent** (§5): overrides are bare + `ConfigValue`s; `describe-setting`'s "Source:" names where `define()` + ran. The inspection view sketched above is currently impossible to + render. +- **Adoption is five settings**: `editing.auto-pair` (pair.lua), + `editing.trim-on-save` (editops.lua), `autosave.interval-ms` + (autosave.lua), `window.panel-height` + `window.min-height` + (window.lua). Everything else a user might set — theme, fonts, LSP + server config, killring size, recentf/saveplace/desktop enables, + pair sets, comment strings, `pmacs.parse.*` — lives in raw Lua + outside the registry and is therefore invisible to `describe-setting` + and any future settings UI. The migration list is already written: + `docs/config-registry-framing.md` "named deferrals" (table-valued + settings are the hard prerequisite for LSP/pair/comment tables). +- **No persistence**: settings changed at runtime do not survive + restart (the `custom-file` split-brain question is a named deferral). +- The three-level separation holds in principle today (registry / + hooks+keymaps / packages), but with five settings registered, level 1 + is effectively empty — users need executable Lua for nearly every + ordinary preference, which is the exact failure the section warns + about. + +--- + +## 12. Profiles as Product-Level Bundles + +Pmacs should offer a small number of official profiles bundling default +keymaps, visible interface regions, package recommendations, settings, +task conventions, discovery hints, and onboarding: **Pmacs Standard** +(approachable graphical workstation), **Emacs** (familiar bindings, +minibuffer-centered), **Minimal**, and later **Research Workstation** +(terminals, remote machines, Slurm, proof assistants, long-running +builds). Profiles must not create separate products — they exercise the +same registries and primitives. + +### Ground truth + +**Grade: missing.** Not a named concept anywhere in the tree. There is +one hardcoded default: a single 161-line keymap +(`builtin/keymaps/default.lua`) that is already a de-facto hybrid of the +"Standard" and "Emacs" profiles (CUA selection + Emacs kill/yank/isearch +chords). No profile object, no bundle format, no selection mechanism, no +per-profile defaults layer (§11's missing profile scope is the same +gap). Prerequisites: the config profile layer, and enough registry +adoption that a profile has something declarative to set. + +--- + +## 13. Package Experience, Not Merely Package Resolution + +Pmacs already has serious package-resolution machinery. Product +coherence requires a package *lifecycle* experience: search, +installation, updates, disable, reload, uninstall, version inspection, +dependency graph, compatibility warnings, capability declarations, +ownership inspection, error history, active-worker inspection, resource +use, trust state. Installation should work during a running session. +Users should be able to install coherent capability bundles ("Rust +Development") rather than individual packages. Marketplace sequencing: +stable format → ownership/reload lifecycle → in-editor manager → curated +registry → bundles → publisher identity → public marketplace. + +### Ground truth + +**Grade: resolution without lifecycle — the artifact layer is mature, +the lifecycle layer assumes a single author iterating on their own +package.** + +**Mature (keep):** `pmacs.toml` manifest (validated name, semver, +`pmacs_required`, dependencies/conflicts, entry, exports); git-address +installs (`github:`/`gitlab:`/URL; auth delegated to git config; no +registry service); iterate-to-fixed-point resolver with deterministic +ordering and honest unsatisfiability errors (documented no-backtracking +tradeoff); merged SHA-256 lockfile; per-package `_ENV`; `exports` +enforced by a position-1 searcher; bundled packages through the +identical path. + +**The lifecycle facts:** + +| Operation | State | +|---|---| +| `install` / `install_project` / `install_local` / `update` | exist, **init.lua-only** — `require_init_phase` raises `InitOnlyApi` mid-session; the error text admits there is no CLI equivalent ("restart with an updated init.lua") | +| `reload(name)` | **works in-session and is well-built**: unload hooks → loaded-table invalidation (name + `name.` prefixes) → env clear → re-require | +| `installed()` / `describe(name)` / `load(name)` / `on_unload(fn)` | work in-session; `describe` returns manifest metadata only | +| uninstall / remove | **absent** — the documented procedure is `rm` in a shell (`src/packages/installer.rs:1178-1180`) | +| disable / enable | **absent** — no concept | +| search / list-available | **absent** — no registry, no index; you must already know a git URL | +| inspect contributions | **absent** — `describe` cannot say which commands/hooks/keys/settings a package contributed; no `*packages*` view exists | + +**Structural findings that any lifecycle arc must address:** + +- **The roster is in-memory per session**, rebuilt from `init.lua` + calls. A package on disk that init.lua doesn't `install` is invisible + to `require`/`installed()`. And because `do_install` unconditionally + runs the resolver, **every startup runs `git fetch --prune --tags` + per package** before the idempotent fast-path can trigger — a + first-launch latency and offline-use problem. (The Rust-side + `UpdatePolicy::Frozen` that would fix offline installs is + **unreachable from Lua** — dead code.) +- **Ownership is not tracked.** `SourceLocation` is path attribution, + not package attribution (for `install_local` the path may be the dev + tree, not the install root); nothing indexes registrations by source; + there is no "what did package X register" query and no + bulk-unregister. The correct signal (`CurrentlyLoadingPackage`) + already exists and is ignored by every registrar (§1.1). +- **The teardown surface is incomplete in a way that makes the + documented convention unsatisfiable: `pmacs.hook.remove` does not + exist** (`install_hook_module` exposes define/add/list/run; + `HookRegistry` has no removal method at all). A package that calls + `pmacs.hook.add` leaks a callback on every reload, permanently. The + package-author guide's hand-rolled `OWNED = {}` cleanup pattern + (`docs/package-author-guide.md:379-415`) cannot be followed for + hooks. (Independently rediscovered by the Lean 4 arc scout.) +- **Error attribution exists on exactly one code path** — + `packages.load` wraps require and logs `[package ] load failed` + to `*errors*` — **and nothing in `builtin/` uses it**. Plain + `require` from init.lua attributes only by traceback; a failing + `install` aborts the whole init.lua with no per-package isolation. +- Install-root directory names are the manifest name's last segment, so + same-basename packages collide on disk (knowingly accepted, + `installer.rs:44-50`). + +Sequencing: ownership + `hook.remove` + attribution is the same work as +§10's class 10.2 and is the prerequisite for disable/uninstall/inspect; +in-session install requires reworking the init-phase gate; search/ +bundles/marketplace remain correctly last. + +--- + +## 14. Coherent Workbench Primitives + +Pmacs should resist implementing each subsystem with a custom UI +vocabulary. It should provide a small set of reusable view primitives — +editable text view, virtual list, tree, structured table, inspector, +output channel, diagnostics collection, task/progress view, diff view, +transient selector, contextual popup, side panel, bottom panel, +help/documentation view — and packages should provide structured models +to them. Git status, project files, symbol outlines, package +dependencies, and worker trees should share one tree model with +consistent selection, expansion, filtering, action discovery, mouse and +keyboard behavior, persistence, and accessibility. + +### Ground truth + +**Grade: partial, with the best trajectory of any concern.** + +Primitive-by-primitive against the list above: + +- **Editable text view** ✓ — the buffer itself, everywhere. +- **List** ✓ — **listview is a real shared primitive**, the strongest + coherence asset in the UI layer: references, outline, buffer-list, + and project-search all use it, with a shared buffer-local keymap + idiom (RET/SPC visit, n/p, g refresh, q quit) that is inspectable and + rebindable (§6's counter-example). +- **Output channel** ✓ — the compile-mode `*compilation*` model + (streamed, intercept-read-only, error-rule parsing), reused by grep + and shell-command. +- **Diagnostics collection** ✓ — `DiagnosticStore` + signs + unified + `error.next` source. +- **Transient selector** ✓ — the minibuffer (though its `source` + vocabulary is fixed Rust-side). +- **Contextual popup** ✓ — completion popup, context menu (each a + shadow, §6). +- **Bottom/side panel** ✓ — landed as bottom-panel Stage 1 (#155): + `WindowParams` side/fixed_rows/dedicated, `display = "current" | + "panel"` adopted by listview/compile/terminal, quit-action, divider + drag. Stage 2 (GPU band) pending its own framing. +- **Task/progress view** △ — `*workers*` exists but joins nothing + (§9). +- **Help view** △ — exists twice (§5); needs unification, not + invention. +- **Tree** ✗ — none. The named future consumers (project files, symbol + hierarchy, package dependency graph, worker trees, git status) will + each need it; building it once *before* dired's directory view and + the workers tree harden their own conventions is exactly this + section's point. +- **Structured table / inspector / diff view** ✗ — none. (`describe.*` + tables are the inspector's data model without a view; the + wire-declared `ResourceOffer` family was reserved for diff/blame + sources and remains unproduced.) + +--- + +## 15. Contextual Affordances + +Pmacs should remain excellent for keyboard-driven users while making +capabilities visible to users who do not know their names: a diagnostic +should offer code actions; a test definition run/debug; a Git change +stage/revert/diff; a missing formatter configuration guidance; a symbol +references/rename/definition/documentation; a remote workspace its +location; a long-running task progress and cancellation. Affordances +should invoke ordinary commands, never separate logic paths. + +### Ground truth + +**Grade: weak.** + +What exists: the right-click context menu — 11 items in 4 groups +(edit/symbol/diagnostic/history, `builtin/menus/default.lua:117-142`), +with a closed context vocabulary (`always`/`selection`/`symbol`/ +`diagnostic`, `src/menu.rs:44`) and per-item predicates that *are* +evaluated (unlike command predicates). It correctly invokes ordinary +commands by name. Its limits: right-click only (no keyboard path in), +no key hints on rows, invisible items filtered rather than grayed, and +the unvalidated command references of §5. + +What does not: + +- **Code actions apply the first action blindly** — no picker (a + roadmap "dark matter" item still true at audit). +- **There is no Git integration at all** — no status, stage, diff, + blame, or gutter markers anywhere in the tree (gutter git riders and + the `ResourceOffer` diff/blame family are named deferrals). The Git + affordance list above has nothing to attach to yet. +- No test run/debug affordances (DAP is a future arc, + `docs/dap-debugging-framing.md`). +- No missing-tool guidance affordances (§1.2 — the diagnostic that + *should* say "rust-analyzer not found — install with rustup" says + nothing). +- No remote-location display (§8 — nothing carries a location). +- Task progress/cancellation affordances exist only inside `*workers*` + (§9); a long-running task shows nothing at the point of origin. + +--- + +## 16. Productize the Semantic Frontend Architecture + +The semantic protocol should be visible as a product advantage: native +frontend rendering, frontend-specific typography, high-quality +decorations, efficient incremental updates, accessible semantic +information, multiple simultaneous frontends, stable remote attachment, +frontend experimentation without reimplementing editor semantics. To +preserve coherence: core commands frontend-neutral; stable semantic +identities; explicit capability negotiation; graceful degradation; +layout state separated from semantic state; no frontend becoming the de +facto privileged implementation. + +### Ground truth + +**Grade: strong — the healthiest concern in this document, and most of +its asks are already practiced.** + +- Versioned, negotiated protocol `SUPPORTED=[6..=20]` with deliberate + encoding-breaking bumps, both-frontends support required per bump, + and byte-pin discipline for appended variants (handoff §4). +- Two genuine frontends share the conceptual model; CRDT concurrent + editing with presence across them; remote attach + reconnect. +- **Graceful per-frontend degradation is practiced, not aspirational**: + fold projection is per-frontend (`FrontendView.fold_projection`, + selected from the negotiated `semantic_render` bit) so a grid + frontend collapses folds while a simultaneous GPU session does not + skip lines (#149/#148). +- The GPU frontend exceeds the TUI (minimap, squiggles, typography) + without the TUI losing the model — the "no privileged frontend" rule + is holding under real divergence pressure. + +Remaining, honestly small relative to the section's ambition: capability +negotiation is per-bit rather than a first-class declared capability +set; layout state vs semantic state separation is partial (window layout +is daemon-side; desktop restore under a daemon is unresolved, §2 step +12); and the advantage is invisible as *product* because §17 means +nobody outside the repo can try it. + +--- + +## 17. Distribution Is Part of the Product + +Pmacs should eventually be installable without repository familiarity: +reproducible release builds, Linux and macOS binaries, checksums and +signatures, stable and nightly channels, one-command update, rollback, +protocol- and package-API compatibility reporting. First launch should +create/locate config directories, explain the default profile, identify +optional external tools, and let the user open a project immediately. + +### Ground truth + +**Grade: missing — zero release machinery exists.** + +`.github/workflows/` contains exactly one workflow, `ci.yml`, and it is +test-only (fmt/clippy/test matrix; the only `release` strings in it are +`cargo test --release` flags). No release job, no artifact upload, no +tags-to-binaries path, no checksums, no channels, no update or rollback +mechanism. Installation is `git clone` + `cargo build --release +--workspace --features pmacs/crdt` (README), which additionally requires +knowing the feature-flag matrix (luajit vs lua54 × crdt). Runtime +dependencies (`/bin/sh`, `stty`, git, tar) are documented in the README +and never checked at runtime. First launch creates nothing and explains +nothing (§18) — though by design it also *requires* nothing (§3), which +is the right half to have. + +This concern is independent of every other arc and can start anytime; +until it does, every other coherence improvement is invisible outside +the repository. + +--- + +## 18. Onboarding + +Pmacs needs onboarding that teaches concepts through use: open a +project → command palette → find a file → terminal → inspect a +diagnostic → view workers → change a setting → inspect where it came +from → Lua REPL → redefine a command. That sequence communicates the +whole thesis: already useful, discoverable, visible computation, +explainable settings, programmable internals. It should be an ordinary, +restartable help workspace, not a one-time modal wizard. + +### Ground truth + +**Grade: missing entirely.** + +No welcome buffer, no tutorial, no first-run detection, no cheat sheet +reachable from inside the editor (`docs/keybindings.md` exists on disk +only). `C-h` is `buffer.delete-word-backward`; there is no help prefix +key and no `F1`. The sole discovery affordance is knowing to press +`M-x` (`builtin/keymaps/default.lua:141` — whose own header comment +calls it the "command palette"). The empty `*scratch*` buffer that +greets a new user says nothing (`EditorCore::new` sets an empty +status). + +Note the dependency: five of the ten onboarding steps above currently +lead somewhere broken or invisible (find a file — in flight; inspect a +diagnostic — silent-failure risk; view workers — undiscoverable; +setting provenance — unanswerable). Onboarding is correctly sequenced +*after* the P1/P4 fixes, but the cheap floor — a welcome buffer in +`*scratch*` naming `M-x`, the keybinding cheat sheet as a help buffer, +and a help prefix decision — has no prerequisites at all. + +--- + +## 19. Product Coherence Acceptance Tests + +Pmacs should add acceptance tests that exercise product behavior across +subsystems, complementing (not replacing) subsystem tests: + +- **Installation/first launch** — no config, open a directory, usable + workspace, actionable guidance for missing tools. +- **Command discovery** — search by title and synonym; display + keybinding, provenance, availability; invoke from palette and menu + through the same object. +- **Workspace lifecycle** — multi-root open, servers, terminal, build, + close, restore, ownership cleanup. +- **Worker ownership** — start completion/search/build, inspect, + cancel a parent, confirm child cancellation and UI recovery. +- **Package lifecycle** — install in-session, inspect contributions, + disable, confirm disappearance, reload, uninstall cleanly. +- **Remote execution** — attach to remote daemon, edit optimistically, + remote terminal and server, disconnect/reconnect, coherent state. + +### Ground truth + +**Grade: missing — but the culture that would make them excellent is the +project's strongest process asset.** + +Zero cross-subsystem journey tests exist. Every acceptance suite in the +tree pins one subsystem's contract (superbly — bite-verified, +falsified-by-revert, vacuity-checked). Several of the scenarios above +are currently *untestable* because the behavior doesn't exist (install +in-session, disable, open a directory); the ones that are testable +(first launch, command discovery, worker cancellation, remote +attach/reconnect) could be written today and would immediately pin the +journey against regression. The first coherence acceptance suite should +be the §2 journey itself, growing a step at a time as steps become +real — that is how "the journey is a release gate" stops being +aspirational. + +(Related lesson already in the handoff: `compile_mode_acceptance` +accidentally reads the real user config — an *unintentional* +whole-product test that keeps catching real coherence bugs. That is +evidence this class of test has teeth.) + +--- + +## 20. Recommended Priority Order + +Each priority is annotated with its audited state and whether the gap is +**wiring** (surface over existing machinery — cheap) or **model** (a +missing runtime entity — a real arc). + +### Priority 1: Protect the golden product journey + +Establish the end-to-end workflow; treat regressions as release +blockers. **State: broken at step 3 (§2). Mostly wiring, and unusually +cheap:** directory-argument handling; a find-file surface (in flight, +PR #162); surfacing the LSP spawn failure with guidance (§1.2); a +compile keybinding + `cargo build`/`test` default from the existing +`ProjectKind::Cargo`; a terminal keybinding; a welcome buffer. The +journey acceptance suite (§19) is the ratchet that keeps it fixed. + +### Priority 2: Make workspace and location explicit + +Otherwise project, LSP, remote, task, and persistence accumulate +incompatible ownership models — the audit confirms four have already +diverged (§7). **State: missing; first slice in flight (multi-root LSP +affinity). Model gap:** the Workspace entity (§7), then Location values +(§8). This is the long-lead arc; start it before the fifth and sixth +subsystems grow their own root conventions. + +### Priority 3: Strengthen extension ownership and isolation + +**State: missing; prerequisite-shaped. Model gap, with one bug-sized +prerequisite: `pmacs.hook.remove` does not exist (§13).** The work +unit: registrations carry their owning package (the +`CurrentlyLoadingPackage` signal already exists), removal APIs complete +the set, error attribution becomes default rather than opt-in. This +single arc unblocks §13's disable/uninstall/inspect, §10's class 10.2, +and package-scoped task cancellation in §9. + +### Priority 4: Unify discovery + +**State: substrate without surface. Almost pure wiring — the best +payoff-per-effort in this document (§5):** a dozen interactive commands +over existing introspection, richer M-x rows (the wire pattern already +exists), title/category on `Command`, predicate evaluation, help-layer +unification, a help prefix key. Most of P1's "understand the interface" +and §18's floor ride on this. + +### Priority 5: Finish the workbench convergence + +**State: partial and moving (§14) — bottom panel Stage 1 landed, GPU +band pending; listview proven.** Remaining: the tree primitive (build +it before dired and the worker tree invent two), table/inspector/diff, +help unification. Wiring plus one modest model piece (the tree model). + +### Priority 6: Productize configuration + +**State: foundation only (§11). Model-lite:** value provenance in the +registry, then layering (profile/workspace scopes — depends on P2 for +workspace, §12 for profiles), then adoption migration (table-valued +settings are the hard prerequisite), then persistence. + +### Priority 7: Build package lifecycle UX + +**State: not started; correctly sequenced after P3.** In-session +install (init-gate rework), disable/uninstall over P3's ownership, +`*packages*` view over P5's primitives, then bundles and registry +sequencing per §13. + +### Priority 8: Ship binaries and release channels + +**State: zero (§17). Independent of everything — can start anytime.** +The editor becomes testable by users who are not repository +contributors; every other priority's value is invisible until this one +exists. + +### How this maps to arcs + +Candidate arc cuts, honoring one-feature-one-branch-one-PR and the +framing workflow (each needs its own scout + framing before any +implementation — this list is direction, not commitment): + +1. **Journey Stage 1** (P1): directory open + compile defaults + + LSP-failure surfacing + bindings + welcome buffer + the first + journey acceptance suite. Rides alongside the in-flight dired arc. +2. **Discovery surface** (P4): the describe/list/where-is command + family, M-x rich rows, help unification, help prefix. +3. **Transient keymap layer** (§6): the overlay scope + lifetime + handle + derived `dispatch_idle`, then migrate shadows one per PR. +4. **Extension ownership** (P3): `hook.remove`, owner-carrying + registrations, attribution-by-default. +5. **Worker identity** (§9): owner/purpose/parent on jobs and + processes, join the four planes, statusline activity indicator. +6. **Workspace entity** (P2): the object, then location values. +7. **Config provenance + adoption** (P6). +8. **Package lifecycle** (P7, after 4). +9. **Distribution** (P8, anytime). + +A standing process change accompanies all of them (§1.3): **every new +framing doc must state its coherence impact** — which journey steps it +touches, whether it adds an interaction island, whether its options +enter the config registry, whether its background work is attributed — +so the debt stops compounding silently. + +--- + +## 21. What Pmacs Should Borrow + +Proven adoption-cost reducers from successful modern editors, with +audited status: immediate usefulness (△ — editing yes, journey no); +strong defaults (✓ where they exist, §3); progressive disclosure (✗ +inverted, §4); searchable commands (△ names-only, §5); integrated +language tooling (✓ data layer / △ surface); project awareness (△ +conventions, §7); visible contextual actions (△ §15); coherent +task/terminal integration (✓ mechanics / ✗ visibility, §9); package +discoverability (✗, §13); configuration layering (△ foundation, §11); +remote development as core workflow (△ works, unmodeled, §8); smooth +distribution and updates (✗, §17); consistent interface primitives (△ +best trajectory, §14); explicit missing-tool guidance (✗ except +`--gpu`, §1.2). + +--- + +## 22. What Pmacs Should Preserve and Deepen + +Pmacs should not trade away the qualities that justify its existence — +and the audit confirms these are today's genuine strengths: live +programmability (redefine/unregister at runtime, per-package envs); +implementation inspectability (SourceLocation on every registration, +mandatory descriptions); replaceable interaction models (aspirational — +§6 is the gap); multiple genuine frontends and semantic rendering (✓, +§16 — the strongest concern); explicit parallel work with +cancellability and observability (mechanics ✓, product visibility ✗, +§9); remote daemon architecture (✓); user control over the editor as a +running system (✓). + +The goal is not to make pmacs less powerful so that it becomes +approachable. The goal is to make power **progressively available**. + +--- + +## 23. Product Thesis + +Emacs offers: *the editor is a programmable environment, and the user +may transform it completely.* VS Code offers: *the editor is already a +coherent development workstation, and extensions fill in the remaining +gaps.* Pmacs should offer: + +> **The editor is already an excellent workstation, and every part of +> that workstation remains inspectable, programmable, concurrent, and +> replaceable.** + +Its strongest distinctive proposition is not "Emacs in Rust" or "Emacs +with threads": + +> **Pmacs is a live-programmable editor in which computation, +> interfaces, ownership, and execution locations are explicit — allowing +> local, remote, interactive, and background work to coexist without +> freezing or becoming opaque.** + +The audit's one-line verdict on the thesis: **"without freezing" is +delivered; "without becoming opaque" is not yet true** — for the +failures a new user meets first (§1.2), for background work (§9), for +settings (§11), and for what a key will do while a modal surface is +active (§6). Product coherence is what will make the architecture +perceptible. Without it, pmacs risks becoming an impressive collection +of subsystems. With it, pmacs becomes a workstation whose complexity is +available without being imposed. + +--- + +## 24. Known documentation drift (as of 2026-07-25) + +Found during the audit; fix opportunistically, ideally before this +document is wired into CLAUDE.md/AGENTS.md as required reading: + +- `docs/keybindings.md` — every `src/editor.rs` line citation in §3 is + stale by ~250–1000 lines despite a "last verified @ `f8096ff` + (2026-07-20)" stamp; its shadow list also omits the terminal `C-c` + escape (reports 5 shadows, actual 6). +- `builtin/api/packages.lua` (EmmyLua annotations) — missing + `install_local`, `reload`, `load`, `describe`, `on_unload`; claims + `update` is unimplemented (it is implemented). +- `CHANGELOG.md` (~line 300) — claims a `describe-key` command for + self-introspection; no such command ever shipped (the Lua API + `pmacs.describe.key` exists; the interactive command does not). +- `docs/config-registry-framing.md` (~658) — claims `describe-setting` + renders through `src/help.rs`; it hand-builds its own text in + `builtin/commands/default.lua`. +- `src/workers_buffer.rs` module doc — says the completions ring caps + at 32; `COMPLETED_RING_CAP` is 64. +- `src/command.rs` doc comment on `predicate` — describes palette + gray-out behavior (T M2.7) that never shipped. + +--- + +## 25. Update protocol for this document + +- **When a PR changes any audited claim here, updating this file rides + that PR** — flip the grade, rewrite the fact, note the PR number. + Same discipline as `docs/agent-handoff.md`. +- Line numbers are hints; symbols are authoritative. When touching a + section anyway, re-verify its citations; do not let this document + accumulate the drift §24 catalogs in others. +- Grades change only with evidence (a landed PR, a re-audit), never + aspirationally. +- The **Ground truth** subsections are a snapshot dated 2026-07-25. If + a future comprehensive re-audit is performed, update the date in the + header and prune superseded facts rather than appending — this is a + briefing, not a log. +- Framing docs for coherence-affecting work should cite the section + they serve (e.g. "COHERENCE §6") and state their coherence impact per + §20's standing process change. diff --git a/builtin/commands/default.lua b/builtin/commands/default.lua index 04c49fe..2a13c21 100644 --- a/builtin/commands/default.lua +++ b/builtin/commands/default.lua @@ -611,6 +611,129 @@ cmd { name = "editor.switch-buffer", } end } +-- find-file (dired arc Stage 0; docs/dired-framing.md Q#DR11) ---------------- +-- +-- Until now pmacs had no discoverable way to open a file by path: a file +-- entered a session only from the CLI, an LSP jump, a project-search +-- visit, or `C-x C-r` (whose prompt does pass free text through, but +-- completes only over the recent list). This is that surface. +-- +-- Two substrate facts shape it, and both are load-bearing: +-- +-- 1. COMPLETION IS FLAT. `source = "files"` lists ONE directory and +-- yields bare basenames (`minibuffer.rs` `list_directory`), capped at +-- the shared candidate limit. A custom function source could not do +-- better: sources are called with NO arguments, so a callback cannot +-- see the input to re-root on, and it runs synchronously outside any +-- coroutine, where `Handle:await()` raises --- so it cannot list a +-- directory either. Hierarchical completion is a named Rust change in +-- the framing, not something this command can fake. +-- +-- 2. A SELECTED CANDIDATE SHADOWS TYPED TEXT. `recompute_candidates` +-- sets `selected = Some(0)` whenever the candidate list is non-empty, +-- and `resolve_accepted_value` returns the CANDIDATE whenever +-- anything is selected. So `on_accept` receives typed text only when +-- the input filters every candidate away --- which, since candidates +-- are basenames and the filter is a subsequence match, is exactly +-- when the input contains a `/`. That makes the deeper-path case work +-- (`sub/inner.txt` matches no basename, so it arrives verbatim) and +-- leaves TWO documented consequences, each pinned by a test rather +-- than left to be rediscovered: +-- +-- (a) typing a NEW bare name that happens to be a subsequence of an +-- existing entry opens the existing file instead of creating the +-- new one --- `find_file_selected_candidate_shadows_typed_text`. +-- A new bare name that matches nothing is unaffected and creates +-- normally (`find_file_bare_new_name_creates_in_the_root`). +-- (b) accepting on EMPTY input opens the first candidate in sort +-- order. `fuzzy_score` returns `Some(0)` for an empty needle, so +-- everything ties and `filter_and_sort` falls back to +-- lexicographic order --- which puts dotfiles first, and can put +-- a DIRECTORY first, in which case the open fails and reports. +-- This is the same mechanism `M-x` and `switch-buffer` already +-- have, so it is inherited rather than introduced; it is recorded +-- as decided, not overlooked, and listed in the framing's +-- deferrals beside the accept-semantics fix that would close it. +-- +-- The root is the active buffer's directory, or the process cwd when the +-- buffer has no backing path (`source_root` defaults to "." Rust-side, +-- so the nil case needs no special handling here). It appears in the +-- prompt because the field itself must stay empty: any prefill would +-- contain a `/` and filter every candidate away, killing completion. + +-- Directory part of a path. "/a/b" -> "/a"; "/a" -> "/"; "a" -> nil. +local function find_file_dirname(path) + local dir = path:match("^(.*)/[^/]*$") + if dir == nil then return nil end + if dir == "" then return "/" end + return dir +end + +-- Expand a leading `~` component using $HOME: `~` -> $HOME, `~/x` -> +-- $HOME/x. `~user` is left alone (no passwd lookup), matching the core's +-- own `expand_tilde`. +-- +-- This has to happen HERE, before the path reaches the core, because +-- `get_or_load_buffer` normalizes the path it STORES but loads from the +-- raw one --- so a `~/...` path deduplicates against an already-open +-- buffer yet fails to load when the file is not open yet. Expanding up +-- front makes both halves agree. +local function find_file_expand_tilde(path) + local home = os.getenv("HOME") + if home == nil or home == "" then return path end + if home:sub(-1) == "/" then home = home:sub(1, -2) end + if path == "~" then return home end + local rest = path:match("^~/(.*)$") + if rest == nil then return path end + return home .. "/" .. rest +end + +-- Turn an accepted value into a path. The value is either a bare +-- basename (a selected candidate) or whatever the user typed, so a +-- non-absolute value joins onto the prompt's root --- which resolves +-- both cases to the same file when they name the same one. +local function find_file_resolve(root, value) + local path = find_file_expand_tilde(value) + if path:sub(1, 1) == "/" then return path end + local base = root or "." + if base:sub(-1) == "/" then return base .. path end + return base .. "/" .. path +end + +-- The active buffer's directory, or nil when it has no backing path. +local function find_file_root() + local buf = pmacs.window.buffer() + if buf == nil then return nil end + local ok, path = pcall(function() return buf:path() end) + if not (ok and path) then return nil end + return find_file_dirname(path) +end + +cmd { name = "find-file", + description = "Open a file by path, completing within one directory.", + fn = function() + local root = find_file_root() + pmacs.minibuffer.read { + prompt = "Find file (" .. (root or ".") .. "): ", + source = "files", + source_root = root, + history = "find-file", + on_accept = function(value) + if value == nil or value == "" then return end + local path = find_file_resolve(root, value) + -- A path that does not exist yet CREATES a buffer bound to + -- it: `display_file` routes through `resolve_target_buffer`, + -- which on NotFound creates, binds, and sets "[new file]". + -- That is Emacs parity and deliberate, so only a real + -- failure (a directory, a permission error) reaches here. + local ok, err = pcall(pmacs.window.display_file, path, { select = true }) + if not ok then + pmacs.editor.set_status("find-file: " .. tostring(err)) + end + end, + } + end } + -- Command palette (M-x) ------------------------------------------------------ -- -- Opens the minibuffer with a "commands" completion source, then diff --git a/builtin/keymaps/default.lua b/builtin/keymaps/default.lua index 7dfb9db..c260170 100644 --- a/builtin/keymaps/default.lua +++ b/builtin/keymaps/default.lua @@ -148,6 +148,7 @@ bind("C-x o", "window.focus-next") bind("C-x O", "window.focus-prev") bind("C-x 0", "window.close") bind("C-x 1", "window.close-others") +bind("C-x C-f", "find-file") bind("C-x b", "editor.switch-buffer") bind("C-x C-b", "editor.list-buffers") bind("C-x ", "editor.next-buffer") diff --git a/docs/active-work.md b/docs/active-work.md index c3fd5f9..b426f38 100644 --- a/docs/active-work.md +++ b/docs/active-work.md @@ -1,6 +1,6 @@ # Active work — cross-machine resume ledger -**Snapshot: 2026-07-24.** This file records volatile work that has not +**Snapshot: 2026-07-25.** This file records volatile work that has not landed on `main`. Read it after `docs/agent-handoff.md`. Remove completed entries when their PR merges; do not let this become a second permanent backlog. @@ -188,6 +188,45 @@ If it does not, stop and repair the remote/fetch configuration. suites**; `git diff --check` clean. The sweep needs an isolated `XDG_CONFIG_HOME` and `-- --skip basedpyright`. +## Dired lane — framing APPROVED; Stage 0 MERGED, Stage 1 next + +- Approved framing: `docs/dired-framing.md` (revision 5), landing as its + own docs PR off `githubsucks/main` @ `2af1ab3`, branch + `githubsucks/dired-framing`, worktree `../pmacs-dired-framing`. The + repo's `-framing`-branch convention (`vterm-framing`, + `gpu-initial-target-framing`, `tab-width-parity-framing`). +- **Stage 0 (`C-x C-f` find-file) MERGED as #162** (`main` @ `2af1ab3`, + 2026-07-25, one review round, 12/12 CI green). Durable facts moved to + `docs/agent-handoff.md` §1 per rule 3 below. +- **Stage 1 (the dired view) is next and unstarted.** Branch `dired` + (worktree `../pmacs-dired-arc`) carries the framing commits only and is + based on the now-superseded `0827dd1`; **rebase it onto the `main` + resulting from the framing PR before implementing**, or cut a fresh + branch — its framing commits become redundant once the docs PR lands. +- Stage 1's scope, from the framing §10: `builtin/runtime/dired.lua`; the + `dired` major mode + mode keymap; buffer-per-directory with lexical + canonicalization and the ownership check; read-only intercept + + `set_round_trip_input`; visit routing through `window.display_file`; + parent/sort/revert/quit; `C-x d` (with the `display` opt) / `C-x C-j`; + cursor preservation by basename; the `dired.kill-when-opening` config + key; **and the tolerant `read_dir` opt** — the only Rust in the stage. +- The one Rust change is load-bearing and is why Stage 1 is not + pure-Lua: `read_dir_blocking` (`src/fs.rs:201`) fails the **entire + listing** on any of five per-entry conditions, and the tolerant wrapper + its own module doc delegates to package authors **cannot be written in + Lua** — the primitive returns one error and no partial vec. +- Coherence (framing §0.5, required since #163): serves `COHERENCE.md` + §20 Priority 1, which names this work explicitly; journey steps 7 and + (partially) 3; **adds no interaction island** — keys are a mode-scoped + keymap, and wdired is a mode swap; adopts `pmacs.config` for + `dired.kill-when-opening`; inherits §9's worker-attribution gap for its + `read_dir` jobs without worsening it. +- **Boundary with the Journey Stage 1 arc** (`COHERENCE.md` §20 arc-cut + 1): CLI directory-argument handling (`pmacs .` exits 1) belongs there, + not here. The two meet at `resolve_target_buffer`; dired supplies the + buffer a directory should resolve *to*, and `pmacs .` should route into + it rather than growing a second directory surface. + ## Bottom-panel lane (window placement + side windows) — Stage 1 IN REVIEW - Portable branch: `githubsucks/bottom-panel`, worktree diff --git a/docs/agent-handoff.md b/docs/agent-handoff.md index fbdf4df..a844230 100644 --- a/docs/agent-handoff.md +++ b/docs/agent-handoff.md @@ -1,7 +1,12 @@ # Agent handoff — cross-machine continuity -**Last updated: 2026-07-24, after GPU initial-target (#148, protocol v20) -landed, following folding Stage 2 (#149) and its landed-doc refresh (#150), +**Last updated: 2026-07-25, after find-file (#162) landed — the dired +arc's Stage 0 — following COHERENCE.md (#163), Lean 4 Stage 1 (#160), the +minimap blank-slab fix (#159), bottom-panel Stage 1 (#155), the +inline-math re-scout (#154), the vterm PTY-flake fix (#153), and the +GPU initial-target doc refresh (#152); and before that GPU +initial-target (#148, protocol v20), +following folding Stage 2 (#149) and its landed-doc refresh (#150), web grammars HTML+CSS (#146), the LaTeX Stage 1 / inline-math framing pair (#144/#145), folding Stage 1 (#142), one-command GPU invocation (#141), the documentation refresh (#140), Vterm Stage 3 (#135), tab-width rendering @@ -19,9 +24,55 @@ reads it the way you just did. For volatile branches, checkpoints, verification, and recovery commands, read `docs/active-work.md` immediately after this file. -## 1. Where the project stands (2026-07-24) +## 1. Where the project stands (2026-07-25) -- `main` @ `0dd16a5` (GPU initial-target #148 atop folding Stage 2 landed-doc +- `main` @ `2af1ab3` (find-file #162 atop COHERENCE.md #163, Lean 4 Stage 1 + #160, minimap blank-slab #159, bottom-panel Stage 1 #155, inline-math + re-scout #154, vterm PTY-flake #153, and doc refresh #152). Protocol + unchanged at **v20**. The bullets below describe the arcs in their own + terms; this line is the head-of-`main` anchor. +- **`COHERENCE.md` is now required reading and a required framing input + — #163.** It carries the product-coherence thesis, an audited + scorecard, per-concern gaps, and §20's priority order, and it is the + standard new work is evaluated against. Per `CLAUDE.md`, **every new + framing doc must state its coherence impact** — journey steps touched, + interaction islands added, config-registry adoption, background-work + attribution. Its §2 grades the golden journey **broken at step 3** + (`pmacs .` exits 1). +- **find-file LANDED — #162** (`docs/dired-framing.md` §10, Q#DR11; merge + `2af1ab3`; one review round). `C-x C-f` is the dired arc's **Stage 0**: + pmacs previously had no discoverable way to open a file by path — no + such command existed and `pmacs.buffer.find_or_open` had no interactive + caller. Pure Lua in `builtin/commands/default.lua`, one keymap line, an + 8-test dispatch-driven acceptance suite; no Rust, no protocol change. + Two substrate facts it documents, both worth knowing before touching + any minibuffer prompt: + - **Completion over files is flat and cannot be made hierarchical from + Lua.** A custom `source` function is called with **zero arguments** + (`minibuffer.rs:591`) and runs synchronously outside any coroutine, + where `Handle:await()` raises — so it can neither see the input to + re-root on nor list a directory. Only the Rust + `CompletionSource::Files { root }` can list, and it is + single-directory and 1024-capped. + - **A selected candidate SHADOWS typed text.** `recompute_candidates` + sets `selected = Some(0)` whenever the list is non-empty + (`minibuffer.rs:372-377`) and `resolve_accepted_value` returns the + candidate over the typed contents (`:564-574`). So free-text accept + fires only when the input filters every candidate away — for + basename candidates under a subsequence filter, when it contains a + `/`. This applies to `M-x` and `switch-buffer` too. Consequences are + pinned as decisions, including the hole where a new bare name that is + a subsequence of an existing entry opens the existing file, and the + empty-input case (`fuzzy_score` gives `Some(0)` for an empty needle + and ties break lexicographically, so dotfiles lead). + - Also: `get_or_load_buffer` computes a normalized path but **loads + from the raw one** (`editor_core.rs:842-856`), so a `~/…` path dedups + against an open buffer yet fails to load one that is not open — + find-file expands the tilde Lua-side. Loading through the normalized + path is a named deferral. +- **GPU initial target LANDED — #148** + (`docs/gpu-initial-target-framing.md` rev 3; merge `0dd16a5`; two review + rounds). `pmacs --gpu [--socket NAME|PATH] FILE` transports exact Unix path refresh #150, folding Stage 2 #149, ledger refresh #147, web grammars #146, LaTeX Stage 1 #144 / inline-math framing #145, and folding Stage 1 #142), protocol **v20** (`SUPPORTED=[6..=20]`; v16 = `ThemeFacts`, v17 = diff --git a/docs/dired-framing.md b/docs/dired-framing.md new file mode 100644 index 0000000..87b4573 --- /dev/null +++ b/docs/dired-framing.md @@ -0,0 +1,1209 @@ +# Dired — framing + +**Revision 5 — 2026-07-25. Status: APPROVED; Stage 0 MERGED as #162.** +Rev 1 passed a ground-truth review; rev 2 fixed round 1's seven findings; +rev 3 fixed round 2's six and was approved; rev 4 recorded what Stage 0's +implementation falsified in the approved text (§0); rev 5 adds the +**coherence impact** statement now required of every framing +(`CLAUDE.md`, `COHERENCE.md` §20) — see §0.5. Deliberately +unnumbered: the roadmap's Arc 8 is GPU +structural parity but `docs/lean4-mode-framing.md` also claims Arc 8, so +the arc space is already forked in uncommitted work. (Rev 2 also cited +`docs/dap-debugging-framing.md` as part of that fork — wrong: its Arc 7 +*matches* the roadmap's Arc 7 = Debugging. R2-5.) Numbering this one +would mint a third claim; it is ranked when the roadmap is next +reconciled. Not on `docs/roadmap-2026-07.md` and not in +`docs/side-quest-backlog.md` — this framing proposes the work as well as +its design. + +## 0. Revision history + +### Round 1 (rev 1 → rev 2) + +- **F1 (load-bearing).** §3's rationale for freezing the M8 fixture was + **false**. Rev 1 claimed the 47 tests pin the package system — + `install_local`, `on_unload` unregistration, `DuplicateName`, per-package + `require` scoping. Verified: `install_local` appears only in the shared + `editor_with_dired()` harness and in doc comments, never in an + assertion; `on_unload`, `DuplicateName`, and require-scoping are + asserted **nowhere** in either file; `dired_source_size_under_audit_ceiling` + is a `lines < 1500` lint. Exactly **one** of 47 + (`dired_package_reload_is_safe_after_init_complete`, + `m8_2_acceptance.rs:1190`) exercises package mechanics; the other 45 + assert dired/wdired behavior. §3 rewritten on honest grounds; the + "shrink the fixture" follow-up repositioned from *if drift appears* to + **scheduled after Stage 3**, because it is now known to be cheap. +- **F2 (load-bearing).** `RET` must not use bare `find_or_open`. + `window_panel.rs:373-376` documents why: `find_or_open` "switches the + ACTIVE window in both branches before firing hooks, so a visit to a + previously unopened file would replace a focused panel." A `RET` in a + panel-displayed dired would swallow the panel. Visits now route through + `pmacs.window.display_file` (new Q#DR10), which also dedups by + *normalized* path. §2 gains the window-primitive ground truth it was + missing entirely, and the `dired` command gains the `display` opt that + acceptance 11 was already testing without. +- **F3 (load-bearing).** Stage 0's completion mechanism does not work as + described. A function source is re-called per keystroke but invoked as + `f.call(())` — **zero arguments** (`src/minibuffer.rs:591`) — and the + callback runs synchronously from Rust dispatch, outside any + `pmacs.async` coroutine, where `Handle:await()` raises + (`async.lua:76-79`). There is no synchronous directory listing in Lua, + so a function source **cannot descend into directories**. Stage 0 + rewritten around the existing Rust `CompletionSource::Files` + (`minibuffer.rs:589`) plus free-text accept; hierarchical completion is + a named deferral (new Q#DR11). Rev 1's "a file you have never opened is + unreachable" is corrected to *undiscoverable and uncompleted* — recentf's + prompt already passes free text to `find_or_open`. +- **F4 (design gap).** Q#DR5(a) as specced did not cover dired's own hard + case. `apply_resource_op`'s rebind uses `find_by_path` + (`buffer_registry.rs:168-174`) — **exact `Path` equality, first match + only** — so a *directory* rename strands every buffer beneath it, and + `R` on a directory line is an ordinary dired operation. Worse, that arm + looks up with the **raw** path (`mod.rs:3248`) while stored paths are + normalized on write (`EditorCore::set_buffer_path`, `editor_core.rs:819`) + and the normalizing wrapper `find_buffer_for_path` (`:864-867`) exists and + is bypassed — so a non-normalized old path silently fails to match today. + Q#DR5 widened to include rebind *semantics*, with new evidence: + **`pmacs.fs.rename` has zero production callers**, so changing the + primitive's contract breaks nobody. +- **F5 (spec fix).** Non-UTF-8 **symlink targets** are fatal too + (`fs.rs:227`), and differ in kind from names: the entry's own name is + fine and nothing needs to pass a target back through `rename`. Tolerant + mode now carries `readlink` failures and target-encoding failures in the + per-entry channel; only **names** stay fatal. +- **F6 / F7 (Q#DR2 gaps).** Name-keyed dedup needs a canonical path form + (`/tmp`, `/tmp/`, `/tmp/../tmp` would mint three buffers), and + found-by-name must verify **dired ownership** before painting into a + buffer through `bypass_intercept`. Both folded into Q#DR2. +- **Minors.** Arc number dropped; `C-x C-r` attributed to `recentf.lua:85` + rather than the default keymap; the tolerant opt must **validate** + unknown keys (`supersede_key`, `fs.lua:73-83`, silently ignores them, so + a typo'd `tolerant` would degrade to fatal mode unnoticed); acceptance 13 + carries the fixture's macOS ignore gate (`m8_2_acceptance.rs:211-213`); + and §8 footnotes one per-entry failure that is *already* tolerated — + a failing `metadata.modified()` yields mtime 0 rather than an error + (`fs.rs:463-476`). + +### Round 2 (rev 2 → rev 3) + +- **R2-1.** Q#DR5's "at reply-settle time" named no seam, and the obvious + one is wrong. The fs ops are fire-and-forget-capable — nothing obliges a + caller to `await` or attach `on_complete` — so a rebind implemented + where results are *consumed* (`_take_result`, `mod.rs:6758`) misses any + rename whose handle is never taken: the rename lands on disk and the + buffer is never rebound. The trap survives one layer down, and an + acceptance that awaits would pass while the fire-and-forget path stayed + broken (the pin-through-the-real-path class again). §7 now names the + **main-thread completion drain** `AsyncRuntime::tick` + (`async_runtime.rs:991`) as the seam, unconditional on success — plus a + fact that makes the implementation non-obvious: **rename settles as an + undifferentiated `ReplyKind::FsUnit`**, the same reply chmod and remove + produce (`:1022-1025` maps `Sleep | FsUnit` alike to `JobResult::Unit`; + there is no `Rename` variant). The drain therefore cannot key on the + reply — it must key on the pending job's own `JobKind::FsRename`, and + the job must **retain from/to** so the paths exist at settle. Stage 2's + acceptance includes a **no-await** rename. +- **R2-2.** The `errors` row shape cannot always carry a name. A per-entry + `readdir` iterator error (`fs.rs:215-218`) has no filename — the entry + never materialized, and the error is wrapped with the *parent* path. §8 + makes `name` optional for that arm; the footer counts it without naming + it. +- **R2-3.** §2's `pmacs.window` inventory listed five exports; there are + **eight** — also `display_target` (`:425`), `panel` (`:440`), and + `set_params` (`:544`). The omission was the relevant one: + `display_target` is "the non-side window a visit from a panel should + address", i.e. the mechanism behind `display_file`'s panel-safety. §9's + loosest sentence — directory descent in a panel-displayed dired, left as + "(or `display`, when dired was itself panel-displayed)" — is now + specified, with the dedication question answered. +- **R2-4.** The Lua mirror of `normalize_buffer_path` is a second + implementation of a canonical form — the tab-width-constants class in + miniature. If the mirror and the Rust normalizer disagree on an edge + (`//tmp`, `~` with `HOME` unset, root's trailing slash), dired's + name-dedup and `display_file`'s `find_buffer_for_path` dedup diverge + **silently**: two buffers, no error. §4 now carries a parity obligation, + and records that the "binding whose only caller is dired" argument + undercounts — Q#DR5's fix touches the same normalizer, so a Lua-exposed + canonicalize has at least two consumers by Stage 2. +- **R2-5.** Header nit: only Lean 4 forks the arc numbering; DAP's Arc 7 + matches the roadmap. Conclusion unchanged, evidence corrected. +- **R2-6.** Stage 0 was "recommended first" with no acceptance and no + ruling on nonexistent paths. `display_file` routes through + `resolve_target_buffer` (`editor_core.rs:885-898`), which on + `ErrorKind::NotFound` **creates** the buffer, binds the path, and sets + status `"[new file]"` — Emacs parity, now stated rather than inherited. + §14 gains four Stage 0 acceptance items. + +### Stage 0 implementation notes (rev 3 → rev 4) + +Implementing Stage 0 falsified one thing the approved text asserted, and +the correction belongs here rather than only in the code. + +- **S0-1. Flat completion and free-text accept do not compose the way + Q#DR11 described.** Rev 3 said Stage 0 is "the flat Rust `Files` source + rooted at the current buffer's directory, plus free-text accept", as + though the two were independent and always both available. They are + not: `recompute_candidates` sets `selected = Some(0)` **whenever the + candidate list is non-empty** (`minibuffer.rs:372-377`), and + `resolve_accepted_value` (`:564-574`) returns the **selected candidate** + in preference to the typed contents. So typed text reaches `on_accept` + **only when the input filters every candidate away** — which, since + candidates are bare basenames and the filter is a case-insensitive + subsequence match, means *when the input contains a `/`*. + Consequences, all now pinned by `tests/find_file_acceptance.rs`: + - the deeper-path case works (`sub/inner.txt` matches no basename, so + it arrives verbatim) — which is what acceptance 0b actually tests; + - the new-file case works **only for names containing a separator**, so + acceptance 0c uses one; + - and there is a genuine hole: typing a **new bare name that is a + subsequence of an existing entry** opens the existing file instead of + creating the new one. `find_file_selected_candidate_shadows_typed_text` + pins that as a decision rather than an accident. + + Closing the hole needs a Rust change to accept semantics — prefer typed + text over the selection when the two differ and the user has not + explicitly moved the selection — which would change `M-x` and + `switch-buffer` too, and so is deliberately **not** made in Stage 0. It + joins the hierarchical-completion deferral in §13. +- **S0-4. Accepting on empty input opens the first-sorted candidate.** + A consequence of S0-1 with an empty needle: `fuzzy_score` returns + `Some(0)` for every entry (`minibuffer.rs:637-640`) and + `filter_and_sort` breaks the resulting tie lexicographically (`:678`), + so an immediate RET opens whatever sorts first — dotfiles lead, and a + directory can lead, in which case the open fails and reports (S0-6). + `M-x` and `switch-buffer` share the mechanism, so this is inherited + rather than introduced; it is documented at the command and listed in + §13 beside the accept-semantics fix that would close it. +- **S0-5. Minibuffer history stores the pre-join value.** `accept` + pushes the resolved value into the history bucket **before** `on_accept` + joins it onto the root, so a `C-p` recall of a root-relative entry + under a different root resolves somewhere else. Rust-side, so Stage 0 + cannot fix it; §13. +- **S0-6. The failure arm is a real path and is pinned.** Accepting a + *directory* candidate reaches `display_file`, whose load fails + (`File::open` on a directory succeeds; the read returns EISDIR), so the + command's `pcall` turns it into a status message instead of letting the + error escape mid-dispatch. + `find_file_accepting_a_directory_reports_instead_of_raising` pins it + through the real accept path and fails when the `pcall` is removed. +- **S0-2. The prompt field must start empty.** Emacs prefills find-file's + field with the directory. Here any prefill contains a `/`, which by + S0-1 filters every candidate away and silently disables completion — + so the root is named in the *prompt string* instead, and the empty + field is pinned by acceptance 0d. +- **S0-3. A leading `~` must be expanded before the path reaches the + core.** `get_or_load_buffer` (`editor_core.rs:842-856`) computes a + normalized path but calls `load_file` with the **raw** one (`:847`), so + a `~/…` path deduplicates against an already-open buffer (dedup goes + through the normalizing `find_buffer_for_path`) yet fails to load a + file that is not open yet. Stage 0 expands the tilde in Lua, which + makes both halves agree without changing core load semantics for the + CLI, LSP, and bootstrap callers. **Using the normalized path for the + load is the better fix and is now a named deferral** (§13) — it is the + same normalize-before-lookup family as Q#DR5's `apply_resource_op` + correction. + +## 0.5. Coherence impact (`COHERENCE.md` §20) + +Required of every framing since #163. This arc was scouted and approved +before that rule existed; the statement is added here rather than +backfilled silently. + +**Section served: §20 Priority 1 — protect the golden product journey**, +which already names this work: *"a find-file surface (in flight, PR +#162)"* and *"directory-argument handling"*. Secondary: §5 (unify +discoverability) and §14 (coherent workbench primitives). + +**Journey steps touched (§2).** + +- **Step 7, "find a symbol or file"** — the file half, which had no + surface at all. Stage 0 (`C-x C-f`, merged as #162) covers opening a + known path; Stages 1–3 cover browsing, which is the half a user + reaches for when they do *not* already know the path. +- **Step 3, "open a real project"** — partially, and the boundary + matters. §2's ground truth grades the journey *broken at step 3* + because `pmacs .` exits 1: `load_file` (`src/file_io.rs:81-87`) does + `File::open` (which succeeds on a directory) then `read_to_end` → + EISDIR, which is not `NotFound`, so `resolve_target_buffer`'s + create-a-`[new file]` arm never fires and the error escapes. + **That is the same mechanism Stage 0 pinned** in + `find_file_accepting_a_directory_reports_instead_of_raising` — where + the `pcall` turns it into a status message instead. Dired Stage 1 is + what makes a directory *open into something* rather than merely fail + politely. + **Boundary with the adjacent arc:** §20's arc-cut list puts CLI + directory-argument handling in "Journey Stage 1", noted as riding + alongside this arc. The two meet at `resolve_target_buffer`. This + framing does **not** claim the CLI path; it supplies the buffer a + directory should resolve *to*, and Journey Stage 1 should route + `pmacs .` into it rather than inventing a second directory surface. +- **Step 4, "understand the visible interface"** — marginally, via the + `dired` major mode showing in the statusline (Q#DR8). +- Steps 1–2, 5–6, 8–12: untouched. + +**Interaction islands added (§6): none — deliberately.** §6 grades this +area "weak, and growing by one island per modal feature", with every +modal surface funnelling through `EditorInstance::dispatch_key`'s +precedence machine. Dired adds no Rust-level interception: its keys are +an ordinary **mode-scoped keymap** through the existing `pmacs.keymap` +registry (Q#DR8), so they are introspectable by `describe.key` and +rebindable like any other binding. Stage 3's wdired is a **major-mode +swap**, not a modal layer — which is the reason Q#DR3 chose a mode swap +over an edit-mode flag. Two existing islands are *consumed* (the +minibuffer prompt for `C-x d`, and Stage 0's), neither added by this +arc. This arc therefore moves §6's count sideways, not up. + +**Config registry adoption (§11): yes.** `dired.kill-when-opening` is +defined through `pmacs.config` with a type, default, and +`mutability = "live"` (Q#DR2), not a bare Lua global — matching the +#127 adopters. Sort mode is deliberately *not* a setting in Stage 1: it +is per-buffer session state, and promoting it would need the +buffer-local scope plus a persistence story the registry does not have +yet (its own named deferral). + +**Background-work attribution (§9): inherited debt, not fixed here.** +Every listing runs as a `pmacs.fs.read_dir` worker job, and those jobs +carry no owner or purpose — §9's gap. A dired refresh will therefore +show up in the activity planes exactly as anonymously as every other fs +job does today. Stage 1 does not fix that and does not make it worse; +when §20's "worker identity" arc lands, dired's jobs are ordinary +consumers of it. Naming it here so the debt is visible rather than +silently compounded (§1.3). + +**Net.** One journey step goes from *no surface* to *a surface*; one +more moves from *fails* toward *resolves*; no island added; one setting +enters the registry; one attribution gap inherited and named. + +## 1. Problem and what ships + +Two separate facts collide here, and the second is why this is worth more +than "a file browser would be nice". + +**Fact one: a complete dired already exists, and ships to nobody.** +`tests/fixtures/pmacs-dired/init.lua` is 1,384 lines of Lua implementing +the read-only directory view (T M8.2) and the wdired editable +rename/chmod layer (T M8.3), pinned by **47 acceptance tests** (15 in +`tests/m8_2_acceptance.rs`, 32 in `tests/m8_3_acceptance.rs`). It was +built as one of M8's three "universality proof" packages — the evidence +that a buffer can be a projection of external state. It lives under +`tests/fixtures/`, and `rg pmacs-dired` outside `tests/` returns zero +hits. Nothing installs it; no user can reach it. + +**Fact two: pmacs has no discoverable way to open a file by path.** There +is no `find-file` command and no `C-x C-f` binding. The complete list of +builtin command names contains nothing matching `file` or `open`; +`pmacs.buffer.find_or_open` (`src/lua_bindings/mod.rs:3104`) is a Lua API +with no interactive caller of its own. A file enters a session via the +CLI (`pmacs FILE`, `pmacs --gpu FILE`), an LSP jump, a project-search +visit, or `C-x C-r` recent-files — whose prompt *does* pass free text +through to `find_or_open` (`recentf.lua:74-80`), so an arbitrary path is +technically reachable, but only by typing it blind into a prompt labelled +"Recent file:" with no completion and no discoverability. +`editor.switch-buffer` (`builtin/commands/default.lua:594`) completes over +*already-open buffers* and reports `no buffer: ` for anything else. + +So dired is not a convenience rider on an existing file surface. **Dired +is the file surface.** That reframes both its value and its risk: it is +the first thing a new user needs, and the last place we can afford a +listing that refuses to render. + +**What ships**, staged (§10): + +- **Stage 1 — the dired view.** A builtin `builtin/runtime/dired.lua`: + read-only listing, navigation, `RET` to visit (files open through + `display_file`, directories descend), sort modes, revert, quit, + `C-x d` / `C-x C-j`, a `dired` major mode with mode-scoped keys, cursor + preservation across refresh — plus the one Rust change Stage 1 needs, a + **per-entry-tolerant `read_dir`** (Q#DR6). +- **Stage 2 — marks and operations.** `m`/`u`/`U`/`t`, deletion flags + `d`/`x`, immediate `D`, `R` rename, `C` copy, `+` mkdir — and the three + filesystem primitives that do not exist yet, plus the rename/rebind fix. +- **Stage 3 — wdired.** The editable layer, carrying over the fixture's + hard-won commit logic. + +`find-file` itself (`C-x C-f`) is separable and is Stage 0 (§10). + +## 2. Ground truth (scouted 2026-07-25, `main` @ `e745068`; verified across review rounds 1 and 2; re-verified against `main` @ `0827dd1`) + +**Base note.** `main` moved from `e745068` to `0827dd1` (Lean 4 Stage 1, +#160) between the scout and approval. The diff touches exactly one file +this framing cites — `builtin/runtime/syntax.lua`, which gained a +`lean = "lean4"` modeline alias — and nothing else in the ground truth +below. The only consequence is a line drift: `set_major_mode` is now +`syntax.lua:497`, not `:492`. Every other citation is unchanged. + +### The existing fixture + +- `tests/fixtures/pmacs-dired/init.lua` (1,384 lines) defines eight + commands: `open-line`, `parent`, `sort-name`, `sort-mtime`, + `sort-size`, `wdired-edit`, `wdired-abandon`, `wdired-commit`. It binds + `RET` and `Backspace` **buffer-locally** at open (`:381-388`), paints by + wholesale `buf:replace` behind a `painting` passthrough flag + (`:294-302`), and keys per-buffer handles by linear scan over + `BufferIdLua.__eq` with a liveness compaction (`:53-67`). +- Its wdired layer is the valuable part and is not naive: a + column-classifying `intercept_edit` (`:640-712`), fixed-width perms with + positional validation (`:520-542`), `\\`/`\n`/`\r`/`\t`/`\xNN` filename + escaping with an **exact inverse** so a no-op commit cannot fire a + spurious rename (`:140-211`), field-by-field external-change detection + including `mtime_nsec` (`:854-902`), duplicate-final-name rejection + before any syscall, and a **two-phase rename through unique temp names** + so swaps and chains commit safely (`:1176-1216`). +- **What the 47 tests actually assert (F1).** 45 assert dired/wdired + *behavior*: rendered listing shape, sort order, escaping round-trips, + intercept column rejection, on-disk chmod/rename effects, the two-phase + swap, external-change detection, partial-application reporting. One + (`m8_2:1190`) exercises package mechanics — reload safety after + `set_init_complete`. One (`m8_2:1245`) is a source-line-count lint. + `install_local` appears only in the shared `editor_with_dired()` harness + as a setup precondition and in doc comments; `on_unload`, + `DuplicateName`, and per-package `require` scoping are asserted nowhere. +- **Two of its own stated limitations are now false.** + - `open-line` on a non-directory errors with "requires the + buffer-from-file API (not yet exposed)" (`:948-961`). + `pmacs.buffer.from_file` (`mod.rs:3054`) and `find_or_open` (`:3104`) + both exist and ship. + - The test seam claims "the v0.1 buffer surface doesn't expose + move_to_byte yet, so tests can't reliably position the cursor" + (`:1355`). `pmacs.editor.goto_byte` (`mod.rs:12765`) and + `move_to_line` (`:12526`) landed with editops (#111). +- **A real defect in its model:** navigation mutates `handle.path` and + repaints, but the buffer was named `*dired:*` at creation and + **there is no `pmacs.buffer.set_name`** — the `pmacs.buffer` table + exports exactly `create`, `from_bytes`, `from_file`, `find_or_open`, + `list`, `kill`, `remove`, `on_removed`, `major_mode`, `set_major_mode`, + `set_round_trip_input`, `mark_create`, `add_intercept`, + `remove_intercept`, `apply_resource_op`, and the style-overlay family. + (`Buffer::set_name` exists Rust-side, unexposed.) So after one `RET` the + buffer name names a directory it is no longer showing. Q#DR2 answers this. + +### The filesystem surface + +- `pmacs.fs` is exactly five worker-dispatched ops — `read_dir`, `stat`, + `rename`, `chmod`, `remove` (the complete `_dispatch_fs_*` set) — plus a + Lua-side polling `fs.watch` (`builtin/runtime/fs.lua:226`). **No + `mkdir`, no `copy`, no symlink-create, no recursive remove.** +- **`read_dir` is all-or-nothing, and this is the load-bearing gap.** + `read_dir_blocking` (`src/fs.rs:201`) returns + `Result, FsError>`. **Five** per-entry conditions fail the + **entire listing**: a per-entry `readdir` error, a failed + `symlink_metadata`, a failed `read_link` (`:228-234`), a non-UTF-8 + symlink target (`:227`), and a non-UTF-8 name (`:238`). The module doc + acknowledges the shape and says "dired-class will likely want a + per-entry-tolerant wrapper but that's the package's job, not the + primitive's" (`:196-200`) — **that wrapper cannot be written in Lua.** + The primitive hands Lua one structured error and no partial vec; there is + nothing to be tolerant *with*. Three concrete failure modes, all + ordinary: + 1. a directory readable but not searchable (`r` without `x`) — `readdir` + succeeds, every child `lstat` fails; + 2. a file unlinked between `readdir` and `lstat` — ENOENT, i.e. a plain + refresh of a busy directory (`/tmp`, a build tree) can just fail; + 3. any single non-UTF-8 filename or symlink target in the directory. + One per-entry failure is *already* tolerated: a failing + `metadata.modified()` yields mtime 0 rather than an error (`:463-476`). +- `read_dir` already takes `(path, opts)` and the opts parser + (`supersede_key`, `fs.lua:73-83`) reads only `opts.supersede` and + **silently ignores unknown keys** — signature-natural for Q#DR6's opt, + but a typo'd `tolerant` would degrade to fatal mode unnoticed. +- `chmod` **follows symlinks** (`src/fs.rs:370`; `fs.lua:104`) while + `read_dir`/`stat` use `lstat`. The fixture rejects symlink perms edits + at intercept time for exactly this reason (`init.lua:629-638`) — that + decision carries over unchanged. +- **`pmacs.fs.rename` has zero production callers** — only its own + definition (`fs.lua:126`), `m8_1`/`m8_3` acceptance, and the fixture. +- `pmacs.fs.rename` does **not** rebind an open buffer's path. + `pmacs.buffer.apply_resource_op` (`mod.rs:3206`) — the LSP + workspace-edit applier — does, but by **exact first match**: its + `"rename"` arm calls `reg.borrow().find_by_path(&from)` (`:3248`), which + is exact `Path` equality over insertion order + (`buffer_registry.rs:168-174`). It also uses the **raw** path while + stored paths are normalized on write + (`EditorCore::set_buffer_path`, `editor_core.rs:819`) and the normalizing + lookup `find_buffer_for_path` (`:864-867`) exists and is bypassed. So the + model rev 1 proposed copying is itself subtly wrong, and has no prefix + rebind anywhere. `apply_resource_op` is also **synchronous and blocking + on the main thread**, unlike every `pmacs.fs` op. Q#DR5. + +### Windows, panels, and how anything gets displayed + +- `pmacs.window` exports **eight** functions: `display` + (`window_panel.rs:356`), `display_file` (`:379`), `display_target` + (`:425`), `panel` (`:440`), `quit` (`:453`), `params` (`:499`), + `set_params` (`:544`), and `resize` (`:591`) — alongside the pre-arc + `switch_buffer`. **`display_target` is "the non-side window a visit from + a panel should address"**, i.e. the mechanism that makes `display_file` + panel-safe; it is what Q#DR10 rests on, and it is the reason a file + visit and a directory descent take different routes (§9). +- **`find_or_open` is panel-hostile, by documented design.** + `window_panel.rs:373-376`: `find_or_open` "switches the ACTIVE window in + both branches before firing hooks, so a visit to a previously unopened + file would replace a focused panel before any display policy could + help." `display_file` is the Q#BP11b answer — a side-effect-free dedup + via the **normalizing** `find_buffer_for_path` *before* any I/O, then + destination resolution before the read, so a dedicated origin cannot + force load-before-failure. LSP visits and compile already route through + it. +- `pmacs.listview` (`builtin/runtime/listview.lua`) implements the + disciplines a read-only panel needs: a read-only `add_intercept` + (`:101-104`), `set_round_trip_input` (`:106`), a buffer-local keymap, a + line→item map, `q`-restores-previous, and the `display = "current" | + "panel"` opt-in (`:132-142`). **But** its keymap is a fixed set — + `RET`/`SPC`/`n`/`p`/`g`/`q` (`:76-88`) — with no extension point, and + its read-only intercept is installed once at panel creation and **never + removed** (`:101`), which wdired requires. Three panels already depend + on this module (references, outline, project-search). +- `set_round_trip_input` (`mod.rs:3085`) is what makes single-key bindings + work on a semantic/GPU frontend: while a marked buffer is active + `dispatch_idle` reports false, so optimistic-apply stays off and `d` + reaches the binding instead of landing as a CRDT insert. The same gate + (`dispatch_idle_for`, `editor.rs:818`) *also* disables optimistic apply + for any focused side window (`!window.is_side()`, Q#BP14a), so a + panel-displayed dired is covered twice. +- **There is no real `read_only` buffer flag** — a standing backlog item + (`docs/side-quest-backlog.md`, cross-cutting substrate). The intercept + idiom is what every generated buffer uses today. + +### Minibuffer completion + +- A `source` **function** is re-called on every keystroke + (`recompute_candidates`, `minibuffer.rs:361`) but invoked as + `f.call(())` — **zero arguments** (`:591`). `pmacs.minibuffer.contents()` + exists, but the callback runs synchronously from Rust dispatch, outside + any `pmacs.async` coroutine, and `Handle:await()` explicitly raises + there (`async.lua:76-79`). There is **no synchronous directory listing in + Lua**, so a function source cannot descend into directories. +- `CompletionSource::Files { root }` exists Rust-side + (`minibuffer.rs:589` → `list_directory(root)`), reached from Lua as + `source = "files"` with `source_root` (`mod.rs:13286`). It is flat, + single-directory, capped at 1024 candidates, and **currently used by + nothing outside a unit test**. +- Free text accepts: `resolve_accepted_value` (`minibuffer.rs:564`) + returns the raw typed string when no candidate is selected. + +### Buffers, modes, keys + +- Mode-scoped keymaps exist since #129: `pmacs.keymap.bind { scope = + "mode", mode = "", … }` (`mod.rs:13432`), resolving buffer-local → + mode → global. `syntax.lua:497` is the **only** `set_major_mode` caller + in `builtin/`, firing on `buffer.after-load`; a `buffer.create`d dired + buffer has no path and fires no `after-load`, so nothing contends. +- `find_or_open` on a directory reaches `file_io::load_file` → EISDIR. + Dired must dispatch on `entry.kind` itself. +- **Keybinding space.** `C-x d`, `C-x C-j`, `C-x C-f`, and `C-x C-q` are + unbound **repo-wide**. `C-x C-r` is bound to recent-files in + `recentf.lua:85` (not the default keymap). `C-c ` is fully taken + by LSP (`lsp.lua:2248-2256`); `C-c @` is the folding prefix + (`fold.lua:48-52`); `C-c C-k` is bound **buffer-locally** by compile + (`compile.lua:231`) and async, not globally. + +## 3. Where dired lives (Q#DR1) + +**A builtin runtime module, `builtin/runtime/dired.lua`, written fresh. +The M8 fixture stays exactly where it is, frozen.** + +Builtin rather than a package, because a surface that is the primary way +to open a file cannot be gated on the user installing something, and +because builtin modules get the load-order and config-registry guarantees +a package does not (the `pair.lua`-before-`lsp.lua` precedent). + +**Why the fixture is not promoted.** Rev 1 argued the 47 tests pin the +package system and would be voided by re-pointing. That was false (F1) — +45 of them assert dired behavior, and behavior transfers. The honest +argument is narrower and rests on three things: + +1. **The harness route is itself the proof.** 46 of 47 tests reach dired + through `install_local` + `require`, and that *routing* — a third-party + package, in its own environment, driving buffers, intercepts, marks, + commands, and keymaps — is the M8 universality claim. A builtin loaded + by the runtime demonstrates nothing about packages. Re-pointing the + suites keeps every behavioral assertion and silently deletes the claim + they were written to support. +2. **The builtin diverges structurally**, so the tests cannot transfer + verbatim anyway: the mark column (Q#DR4) shifts every column offset the + wdired tests hardcode through `_test.NAME_START`; mode-scoped keys + (Q#DR8) replace the buffer-local `RET`/`Backspace` binds + `dired_ret_and_backspace_keybindings_navigate` drives; + buffer-per-directory (Q#DR2) replaces the in-place repaint that + `dired_parent_command_navigates_up_one_level` asserts; and the whole + `M._test` seam is package-shaped. +3. **The behavior gets re-pinned regardless**, by the builtin's own + acceptance (§14), which is where those 45 assertions are owed a home. + +The cost is real: roughly 900 lines of rendering, escaping, and commit +logic will exist in two places. The mitigation is that the fixture is +**frozen** — a proof artifact, not a maintained feature, already fully +pinned. + +**Named follow-up, scheduled rather than conditional (F1 corollary).** +Because the fixture's package-system value concentrates in exactly one +test plus the harness routing, shrinking it to the minimum payload that +still proves universality is cheap — far cheaper than rev 1 implied. It +is scheduled **after Stage 3**, when the builtin owns every behavior the +45 tests currently cover, not left to "if drift shows up". + +The fixture remains a *reference* for the parts that were hard — +escaping with an exact inverse, two-phase rename, external-change +detection, the symlink rules — and those carry over as decided design, +not as re-litigated questions. + +## 4. Buffer model and navigation (Q#DR2) + +**One buffer per directory, found-or-created by canonical name; +navigation opens the target's buffer rather than mutating the current +one.** + +This is Emacs's actual behavior (`dired-find-file` on a directory yields a +dired buffer for that directory), and it answers the fixture's stale +buffer-name defect without adding a `pmacs.buffer.set_name` binding: +nothing is ever renamed, because a buffer's name always describes the +directory it was created for. It also makes `C-x d` on an +already-visited directory and `C-x C-j` dedup for free. + +**Canonicalization (F6).** `/tmp`, `/tmp/`, and `/tmp/../tmp` are all +absolute and would otherwise mint three buffers. The rule is **lexical +normalization before naming and before lookup**: expand a leading `~`, +absolutize, collapse `.` and `..`, strip a trailing slash except at root. +Symlinks are **deliberately not resolved** — Emacs parity, and resolving +them would make `..` from a symlinked directory jump somewhere the user +did not navigate from. The core already has exactly this shape in +`normalize_buffer_path` (`editor_core.rs:4790`); it is not Lua-exposed, +so Stage 1 either mirrors it in Lua or exposes it. + +**A mirror is a second implementation of a canonical form, and needs a +parity pin (R2-4).** This is the tab-width-constants class in miniature: +if the Lua mirror and the Rust normalizer disagree on an edge — `//tmp` +(POSIX gives a leading double slash implementation-defined meaning), `~` +with `HOME` unset, root's trailing slash, a `..` that would escape root — +then dired's name-dedup and `display_file`'s `find_buffer_for_path` dedup +**diverge silently**: two buffers for one directory, no error anywhere. +So whichever route Stage 1 takes, it carries a **parity acceptance** that +drives both implementations over one shared edge-case list and asserts +identical output (acceptance 3b). + +Rev 2's "avoids a binding whose only caller is dired" also **undercounts**: +Q#DR5's rename fix touches the same normalizer, so a Lua-exposed +`pmacs.path.canonicalize` (or equivalent) has at least two consumers by +Stage 2. Exposing it and deleting the mirror is therefore the better +end state; Stage 1 may still mirror if exposure turns out to drag in +`EditorCore` borrow plumbing it does not otherwise need, but the parity +acceptance is required either way, and the mirror is then a **named +Stage 2 removal**, not a permanent duplicate. + +**Ownership check (F7).** `pmacs.buffer.create` takes any caller-chosen +name, so a foreign buffer named `*dired:/tmp*` would be "found" by name +and then painted into through `bypass_intercept`. Found-by-name must +confirm the buffer is dired-owned — present in dired's handle table, or +`pmacs.buffer.major_mode(buf) == "dired"` — and otherwise create a fresh +buffer under a disambiguated name rather than clobbering it. + +Buffer name: `*dired:*`. + +The cost is buffer accumulation when walking a deep tree. Emacs users +live with this; Emacs 28 added an opt-out, and we mirror it as a config +key rather than a hardcoded policy: + +- `dired.kill-when-opening` (boolean, default `false`, live) — when true, + descending or ascending kills the dired buffer being left. + +## 5. Read-only discipline and the wdired seam (Q#DR3) + +The dired buffer is read-only by the listview idiom — an `add_intercept` +that rejects every non-bypass edit, plus `set_round_trip_input(buf, +true)` so a GPU session's optimistic-apply cannot swallow single-key +bindings — and dired's own paints use `bypass_intercept = true`. + +**Dired owns its buffer directly; it is not built on `pmacs.listview`.** +Listview would have to grow a keymap extension point and a removable +read-only mode, and three shipped panels depend on it. Bending a module +into a shape its existing callers do not need is exactly the change class +that took CI red in #155: scoping `pmacs.window.buffer()`'s no-arg arm +"for consistency" made a total function partial and silently dropped +edits from six unpcall'd runtime callers. Dired reuses listview's +*disciplines*, not its code. Factoring the shared disciplines into a +common helper is a named follow-up, to be done once dired's real shape is +known rather than predicted. + +**Wdired (Stage 3) is a mode swap, not a flag.** `C-x C-q` removes the +read-only intercept, installs the column-classifying one, and calls +`set_major_mode(buf, "wdired")`; commit and abandon restore both. Because +keys are mode-scoped (Q#DR8), the entire keymap changes with the mode — +`m` means "mark" in dired and means "type an m" in wdired, with no +per-key bookkeeping. + +**Wdired refuses to open on a partially-listed directory.** If the +listing carried any per-entry error (Q#DR6), rename and chmod are refused +with that reason: a rename batch is planned against a snapshot, and you +cannot safely plan against a directory you could not fully see. + +## 6. Marks (Q#DR4) + +Emacs's mark column is column 0, so every other column shifts right by +two (`" "` or `"* "`). The builtin computes its offsets from the +constants rather than inheriting the fixture's `PERMS_START = 1` / +`NAME_START = 39`. + +**Marks are keyed by basename, never by line index.** A sort, a revert, +or an external change reorders lines; a line-indexed mark set would +silently retarget onto a different file — the same class of defect as +keying kill-ring state by index rather than by stable id (the Arc 2 +substrate rule). `*buffer-list*` keys its deletion marks by buffer id +(`builtin/commands/default.lua:507`) for the same reason. + +Two mark characters, following Emacs: `*` (general mark, consumed by +operations) and `D` (deletion flag, consumed by `x`). A basename that +disappears between marking and executing is dropped from the batch and +reported, not silently skipped. + +## 7. Operations and the missing primitives (Q#DR5) + +Stage 2's operations need three filesystem primitives that do not exist, +and one correctness fix. + +**New `pmacs.fs` ops** (worker-dispatched, matching the existing five; +mutating ops take no `supersede`, per `fs.lua:101-124`): + +- `pmacs.fs.mkdir(path, opts)` — `opts.parents` for `create_dir_all`. +- `pmacs.fs.copy(from, to, opts)` — regular files in v1; a directory + source is **refused** rather than silently shallow-copied. Preserves + mode bits; `opts.overwrite` defaults false and the op refuses an + existing target otherwise. +- `pmacs.fs.remove_dir_all(path)` — separate from `remove` rather than a + flag on it, so a recursive delete can never be reached by a caller that + meant the single-object op. + +**The rename rule — decided, with the semantics widened (F4).** A rename +must rebind open buffers, and it must do so at the **primitive**: +`pmacs.fs.rename` has **zero production callers**, so changing its +contract breaks nobody, and leaving the trap armed guarantees the next +caller rediscovers it the expensive way. The rebind is: + +- **prefix-aware**, not exact-match. `apply_resource_op`'s + `find_by_path` (`buffer_registry.rs:168-174`) is exact `Path` equality, + first match only — which strands every buffer beneath a renamed + *directory*, and `R` on a directory line is an ordinary dired + operation. The reconcile rebinds the renamed path itself **and** every + buffer whose path has it as a path-component prefix. +- **normalize-before-lookup.** Stored paths are normalized on write + (`editor_core.rs:819`) and the normalizing wrapper + `find_buffer_for_path` (`:864-867`) already exists; + `apply_resource_op` bypasses it with a raw lookup (`mod.rs:3248`), which + is a latent miss today. The new path goes through the wrapper. + `apply_resource_op`'s own raw lookup is fixed in the same change — it is + the same bug, one call site away. +- in the **main-thread completion drain**, `AsyncRuntime::tick` + (`async_runtime.rs:991`), unconditionally on success — **not** where + results are consumed. This is the load-bearing half of the decision + (R2-1). The fs ops are fire-and-forget-capable: nothing obliges a caller + to `await` or attach `on_complete`, so a rebind hung off `_take_result` + (`mod.rs:6758`) would miss every rename whose handle is never taken — + the rename lands on disk, the buffer is never rebound, and the trap + survives one layer below where we thought we fixed it. An acceptance + that awaits the rename would pass throughout, so **Stage 2's acceptance + includes a no-await rename** and bites against the drain. + + Two facts make this non-obvious to implement. **Rename settles as an + undifferentiated `ReplyKind::FsUnit`** — the same reply `chmod` and + `remove` produce; there is no `Rename` variant, and the drain arm maps + `Sleep | FsUnit` alike to `JobResult::Unit` (`async_runtime.rs:1022-1025`). + So the drain cannot key on the reply; it must key on the **pending job's + own `JobKind::FsRename`**. And the from/to paths live only in the + dispatch call today, so the pending job must **retain them** for the + drain to have anything to rebind with. Both are additive to + `async_runtime.rs`; neither changes the wire or the worker contract. + +**v1 supports `R` on a directory** — that is precisely what prefix-aware +rebinding buys, and refusing it while `C` refuses directory sources for a +different reason (no recursive copy primitive) would be an arbitrary +asymmetry. Stage 2's acceptance pins the directory case explicitly. + +**Confirmation.** Destructive operations (`x`, `D`, recursive delete, +overwriting copy) prompt. There is no `y_or_n` helper — a named +autosave-arc deferral — so Stage 2 adds one rather than repeating +`autosave.lua:219`'s two-element `minibuffer.read` at four call sites. + +## 8. Tolerant listing — the Stage 1 Rust change (Q#DR6) + +`read_dir` grows a per-entry error channel behind an **opt**. The Lua +result under `{ tolerant = true }` becomes: + +```lua +{ entries = { , ... }, errors = { { name = "..." | nil, message = "..." }, ... } } +``` + +with per-entry failures recorded and enumeration continuing. Errors on +the **parent** `read_dir` itself stay fatal — a directory you cannot open +has no partial answer. Dired renders a footer line (`N entries +unreadable`) and refuses wdired (§5). + +**`name` is optional (R2-2).** A per-entry `readdir` *iterator* error +(`fs.rs:215-218`) carries no filename — the entry never materialized, so +there is nothing to name, and the error is wrapped with the **parent** +path. That arm reports `name = nil`; the footer counts it without naming +it. Every other per-entry arm has an entry in hand and names it. + +**What moves into the per-entry channel (F5):** per-entry `readdir` +errors, `symlink_metadata` failures, `read_link` failures +(`fs.rs:228-234`), and **non-UTF-8 symlink targets** (`:227`). A +non-UTF-8 target differs in kind from a non-UTF-8 name: the entry's own +name is fine, the listing renders it with the target shown as unknown, +and nothing needs to pass the target back through `rename`. As it stands +today, one weird symlink in `/tmp` kills the entire listing — the exact +failure class this section exists to fix. + +**Non-UTF-8 names stay fatal**, and are a named deferral. Rendering them +tolerantly is not a listing problem but a *path representation* problem: +`FsDirEntry.name` is `String`, every `pmacs.fs` op takes a `String` path, +and `src/fs.rs:152-155` names byte-preserving paths as post-v0.1 work +that widens the whole surface. Doing it properly changes the type of +every path in the API; doing it improperly hands dired a name it cannot +pass back to `rename`. Stage 1 reports the directory as unlistable with +the offending bytes named, which `FsError::NonUtf8Path` already carries. + +**Why an opt rather than a shape change.** `read_dir` already takes +`(path, opts)`, so it is signature-natural; it keeps the change additive +for third-party packages; and it leaves the frozen fixture's bare-array +consumption (`init.lua:312`) untouched, which matters because a proof +artifact that must be edited to accommodate new work is not frozen. + +**The opts parser must validate (minor c).** `supersede_key` +(`fs.lua:73-83`) reads only `opts.supersede` and silently ignores every +other key, so a typo'd `tolerant` would degrade to fatal mode with no +signal. The tolerant change adds unknown-key rejection to the read ops' +opts parsing. + +**Already tolerated, for the record:** a failing `metadata.modified()` +yields mtime 0 rather than an error (`fs.rs:463-476`), so the per-entry +channel is not the first such concession — it is the first *explicit* one. + +## 9. Keybindings, display, and the major mode (Q#DR7, Q#DR8, Q#DR10) + +**Global** (both unbound repo-wide): + +- `C-x d` → `dired` — prompt for a directory, defaulting to the current + buffer's directory. Takes the standard `display = "current" | "panel"` + opt (Q#BP11b), defaulting to `"current"` in Stages 1–2 like every other + adopter. +- `C-x C-j` → `dired-jump` — dired on the current buffer's file's + directory, cursor seated on that file. + +**Visit routing (Q#DR10, F2).** A `RET` on a **file** line goes through +`pmacs.window.display_file(path, { select = true })`, never bare +`find_or_open`. `find_or_open` switches the active window in both +branches before firing hooks (`window_panel.rs:373-376`), so a `RET` in a +panel-displayed dired would replace the panel with the visited file — +the panel swallows itself. `display_file` is the Q#BP11b answer: it dedups +side-effect-free through the **normalizing** `find_buffer_for_path` +before any I/O, resolves the destination before the read, and is what LSP +visits and compile already use. Underneath, its panel-safety comes from +`display_target` (`window_panel.rs:425`) — "the non-side window a visit +from a panel should address". + +**Directory descent routes differently, and deliberately (R2-3).** A +`RET` on a **directory** line replaces the dired buffer **in the window +dired already occupies**: `switch_buffer` when dired is in a document +window, and `pmacs.window.display(buf, { side = , select = +true })` when dired is panel-displayed. This is Emacs behavior — walking +a tree in a side window keeps the side window — and it is the opposite +routing from a file visit for a principled reason: a *file* is not a +dired buffer and belongs in the document area (hence `display_target`), +while the next *directory* is the same kind of thing as the current one +and belongs in the same slot. **Dedication is a property of the slot, not +the buffer**, so a dedicated dired panel stays dedicated across descent +and the new dired buffer inherits it; Stage 1's acceptance pins that +rather than assuming it, since it is a `Layout`/`WindowParams` behavior +this framing does not otherwise touch. + +**Mode-scoped on `dired`** (Q#DR8: `scope = "mode", mode = "dired"`, +bound once at load rather than per buffer — dired is the first real +consumer of #129's mode keymaps beyond language detection): + +| Key | Command | Stage | +|-----|---------|-------| +| `RET`, `f` | visit (dir → descend, file → `display_file`) | 1 | +| `^` | parent directory | 1 | +| `n` / `p`, `` / `` | move by line | 1 | +| `g` | revert (re-read, preserve cursor and marks) | 1 | +| `q` | quit (restore previous buffer / `window.quit` in a side window) | 1 | +| `s` | cycle sort mode (name → mtime → size) | 1 | +| `m` / `u` / `U` / `t` | mark / unmark / unmark-all / toggle | 2 | +| `d` / `x` | flag for deletion / execute flagged | 2 | +| `D` | delete now (confirms) | 2 | +| `R` / `C` / `+` | rename / copy / mkdir | 2 | +| `w` | copy filename to the kill ring | 2 | +| `C-x C-q` | toggle wdired | 3 | + +**Mode-scoped on `wdired`:** `C-c C-c` commit, `C-c C-k` abandon — +matching compile's buffer-local `C-c C-k` idiom without colliding with +it, since dired buffers are never compilation buffers. + +Everything follows the `M-;` / `M-%` / `C-c @` precedent of shipping the +faithful Emacs default; users rebind through `pmacs.keymap`. + +**Cursor preservation (Q#DR9)** is a Stage 1 requirement, not a nicety: +`g`, a sort, and every Stage 2 operation repaint wholesale, and a dired +that drops you to line 0 after each mark is unusable. The cursor is +re-seated by **basename**, falling back to the nearest surviving line +index when the file is gone — `pmacs.editor.move_to_line` +(`mod.rs:12526`) makes this exact rather than the `move_down`-in-a-loop +walk `listview.lua:68-74` uses. + +## 10. Staging and scope + +- **Stage 0 (separable, recommended first) — `find-file` (Q#DR11).** + `C-x C-f` → `pmacs.minibuffer.read` with `source = "files"` and + `source_root` set to the current buffer's directory, accepting free text + (`resolve_accepted_value`, `minibuffer.rs:564`) into + `pmacs.window.display_file`. **Completion is flat and does not + descend**: a function source cannot list a directory (it is called with + zero arguments and cannot `await`, F3), and the Rust `Files` source is + single-directory and 1024-capped. Typing a full path still works via + free-text accept; typing a *prefix* completes only within the root. + Hierarchical completion is a named Rust change (§13) — either pass the + current input to custom sources, or re-root the `Files` source per + keystroke. **A nonexistent path creates a `[new file]` buffer** rather + than erroring: `display_file` routes through `resolve_target_buffer` + (`editor_core.rs:885-898`), which on `ErrorKind::NotFound` creates the + buffer, binds the path, and sets that status. This is Emacs parity and + is stated rather than inherited (R2-6). Stage 0 carries its own + acceptance (§14) — it is small but no longer trivial to describe + honestly, which is itself an argument for taking it as its own PR. It is + not required by any later stage; Stage 1's `RET` opens files directly. + **Say if you want it folded into Stage 1 instead; it is one branch + either way.** +- **Stage 1 — the dired view. Approval-critical.** + `builtin/runtime/dired.lua`; the `dired` major mode and mode keymap; + buffer-per-directory with canonical naming and the ownership check; + read-only intercept + round-trip input; visit routing through + `display_file`; parent / sort / revert / quit; `C-x d` (with the + `display` opt) / `C-x C-j`; cursor preservation across repaint; the + `dired.kill-when-opening` config key; **and the tolerant `read_dir` opt + plus its unknown-key validation** (Q#DR6) — the only Rust in this stage. + No wire change; no protocol bump. +- **Stage 2 — marks and operations.** The mark column and basename-keyed + mark set; `m`/`u`/`U`/`t`/`d`/`x`/`D`/`R`/`C`/`+`/`w`; + `pmacs.fs.mkdir` / `copy` / `remove_dir_all`; the **prefix-aware, + normalized rename rebind in the completion drain** and the matching + `apply_resource_op` raw-lookup fix (Q#DR5), pinned by a **no-await** + rename and by a directory rename that must not strand the buffers + beneath it; a `y_or_n` confirm helper; and removal of the Lua + canonicalization mirror if Stage 1 shipped one (Q#DR2). +- **Stage 3 — wdired.** `C-x C-q` mode swap; the column-classifying + intercept over the mark-shifted layout; escape/unescape round-trip; + duplicate-name and NUL/slash rejection pre-syscall; two-phase rename; + field-by-field external-change detection; the symlink perms and symlink + target rules; partial-application reporting. +- **After Stage 3 — shrink the M8 fixture** to the minimum payload that + still proves package universality (§3). + +Stages 2 and 3 are sketched here and each gets its own detailed framing +after the prior stage lands, per the folding-arc precedent. **This +framing asks approval for the architecture and Stage 1's detail.** + +## 11. Numbered decisions + +- **Q#DR1** Dired ships as `builtin/runtime/dired.lua`, written fresh; + the M8 fixture stays frozen under `tests/fixtures/` because the + **harness routing** (46/47 tests reaching dired through `install_local` + + `require`) *is* the universality proof and dies if re-pointed, and + because the builtin diverges structurally. Its 45 behavioral assertions + are re-pinned by §14. Shrinking the fixture is **scheduled after + Stage 3**. (§3) +- **Q#DR2** One buffer per directory, `*dired:*`, + found-or-created by name; navigation opens the target's buffer rather + than renaming the current one (there is no `pmacs.buffer.set_name`). + Names and lookups are **lexically normalized** (tilde, absolutize, + `.`/`..`, trailing slash) with **symlinks deliberately unresolved**; + found-by-name **verifies dired ownership** before painting. + A Lua mirror of `normalize_buffer_path` is a second canonical form and + requires a **parity acceptance** against the Rust normalizer; exposing + the normalizer instead is the preferred end state, since Q#DR5 gives it + a second consumer. `dired.kill-when-opening` (default `false`) mirrors + Emacs 28's opt-out. (§4) +- **Q#DR3** Read-only via `add_intercept` + `set_round_trip_input`, with + dired's own paints using `bypass_intercept`; dired owns its buffer and + does **not** extend `pmacs.listview`; wdired is a major-mode swap, and + refuses to open on a partially-listed directory. (§5) +- **Q#DR4** Mark column at column 0 shifts all offsets; marks are keyed + by **basename**, never line index; `*` and `D` are the two mark + characters; a vanished basename is dropped from a batch and reported. + (§6) +- **Q#DR5** Stage 2 adds `pmacs.fs.mkdir` / `copy` / `remove_dir_all`. + Rename rebinding is fixed **at the primitive** (zero production callers + to break), **prefix-aware** (a directory rename must not strand the + buffers beneath it), **normalize-before-lookup** (fixing + `apply_resource_op`'s raw `find_by_path` in the same change), and in the + **main-thread completion drain** `AsyncRuntime::tick` — never in the + take/await path, which a fire-and-forget rename never reaches. Because + rename settles as an undifferentiated `ReplyKind::FsUnit`, the drain + keys on the pending job's `JobKind::FsRename` and the job retains + from/to. `R` on a directory is supported in v1; `C` refuses directory + sources. Destructive ops confirm via a new `y_or_n` helper. (§7) +- **Q#DR6** `read_dir` becomes per-entry tolerant behind an **opt** + (`{ tolerant = true }`), carrying per-entry `readdir`/`lstat`/`readlink` + failures **and non-UTF-8 symlink targets** in an `errors` channel; + parent-level failures stay fatal; non-UTF-8 **names** stay fatal, + deferred to a byte-preserving path surface. The read ops' opts parsing + gains unknown-key rejection. (§8) +- **Q#DR7** Emacs-parity bindings: global `C-x d` (with the `display` + opt) / `C-x C-j`; mode-scoped in-buffer keys per the §9 table; wdired on + `C-x C-q` with `C-c C-c` / `C-c C-k`. (§9) +- **Q#DR8** Keys are **mode-scoped** (`scope = "mode", mode = "dired"`), + not buffer-local — bound once at load, and the wdired swap changes the + whole keymap with the mode. Dired is #129's first non-detection + consumer. (§9) +- **Q#DR9** Cursor is re-seated by **basename** after every repaint, + falling back to the nearest surviving index, via + `pmacs.editor.move_to_line`. (§9) +- **Q#DR10** File visits route through `pmacs.window.display_file` + (panel-safe via `display_target`), never bare `find_or_open`, which + switches the active window before hooks and would let a `RET` replace + the panel dired is displayed in. **Directory descent instead reuses + dired's own window** — `switch_buffer` in a document window, + `display { side = , select = true }` in a panel — because + the next directory is the same kind of thing as the current one. + Dedication is a slot property and follows across descent. (§9) +- **Q#DR11** Stage 0's completion is the flat Rust `Files` source rooted + at the current buffer's directory, plus free-text accept. Function + sources cannot descend (zero-argument call, no `await` in the dispatch + context); hierarchical path completion is a named Rust deferral. + **Amended by S0-1:** the two are not independent — a selected candidate + **shadows** typed text, so free-text accept is reached only when the + input filters every candidate away (in practice, when it contains a + `/`). The prompt field therefore starts empty (S0-2), and a leading + `~` is expanded Lua-side (S0-3). (§0, §10) + +## 12. Bets + +- **B1** The fixture's hard parts — escaping with an exact inverse, + two-phase rename, field-by-field external-change detection, the symlink + rules — transfer to the builtin as decided design. FALSIFIABLE at + Stage 3: if the mark-shifted layout or the mode swap forces a different + commit model, the fixture stops being a reference and Stage 3 is + re-framed from scratch. +- **B2** Tolerant `read_dir` is the *only* Rust change Stage 1 needs — + `display_file`, mode keymaps, `move_to_line`, and + `set_round_trip_input` all already exist. FALSIFIABLE during + implementation; the most likely miss is **scroll** preservation, since + the daemon owns `view_top` and "viewport facts on the wire" is a + standing backlog gap — if preserving scroll (not just cursor) across a + repaint needs a new fact, Stage 1 grows. +- **B3** Mode-scoped single-key bindings survive both frontends, because + `set_round_trip_input` keeps optimistic-apply off — and, when dired sits + in a panel, the `!window.is_side()` arm of the same gate + (`editor.rs:818`) covers it a second time. FALSIFIABLE on a real GPU + session: pressing `d` must flag, never insert. +- **B4** Buffer-per-directory does not produce clutter users complain + about (Emacs parity), and `dired.kill-when-opening` is a sufficient + escape hatch. Falsifiable only by use. +- **B5** Flat, non-descending completion is acceptable for Stage 0 + because free-text accept covers the full-path case. FALSIFIABLE + immediately by use: if typing full paths blind is what people actually + do, hierarchical completion stops being a deferral and becomes Stage 0's + real scope. + +## 13. Deferred (named) + +- **Hierarchical path completion** — pass the current minibuffer input to + custom sources, or re-root `CompletionSource::Files` per keystroke + (Q#DR11). +- **Typed text vs. a selected candidate on accept** (S0-1) — prefer the + typed contents when they differ from the selection and the user has not + explicitly moved it. Closes Stage 0's "a new bare name that is a + subsequence of an existing entry opens the existing file" hole, but + changes `M-x` and `switch-buffer` accept semantics too, so it needs its + own reasoning and gates. **It would also close the empty-input case** + (S0-4): `fuzzy_score` returns `Some(0)` for an empty needle + (`minibuffer.rs:637-640`) and `filter_and_sort` breaks ties + lexicographically (`:678`), so accepting immediately opens the + first-sorted entry — dotfiles first, and possibly a directory, which + then fails and reports. Inherited from the shared minibuffer, not + introduced by find-file, and recorded as decided rather than + overlooked. +- **Minibuffer history stores the pre-join value** (S0-5) — + `Minibuffer::accept` pushes the *resolved* value (a bare basename, or a + root-relative path) into the history bucket Rust-side, **before** Lua + joins it onto the root. So recalling `sub/inner.txt` with `C-p` under a + *different* root resolves against the new root, and can silently create + a `[new file]` buffer somewhere else. Emacs's `file-name-history` stores + absolute paths. Lua cannot fix this — the push happens before + `on_accept` runs — so it belongs with the other Rust-side minibuffer + deferrals here. +- **Load through the normalized path** (S0-3) — `get_or_load_buffer` + computes a normalized path and then loads from the raw one + (`editor_core.rs:842-856`), so tilde paths dedup but do not load. Same + normalize-before-lookup family as Q#DR5's `apply_resource_op` fix. +- **Non-UTF-8 filenames** — needs byte-preserving `pmacs.fs` paths + (`src/fs.rs:152-155`); widens every path in the API. (Non-UTF-8 symlink + *targets* are handled in Stage 1, §8.) +- **Shrinking the M8 fixture** — scheduled after Stage 3 (§3). +- **Factoring the shared panel disciplines** out of dired and + `pmacs.listview` (§5). +- `o` / `C-o` visit-in-other-window — the GPU has no splits (GPU + structural parity, roadmap Arc 8). +- `!` / `&` shell command on marked files; `Q` query-replace across marked + files; `A` search across marked files. +- `i` insert-subdirectory (in-buffer recursive listing) and + `dired-hide-details`. +- Owner and group columns — no uid/gid → name primitive exists. +- Human-readable sizes; sort by extension; reverse-sort toggle. +- `%m` / `%d` regex mark family; `dired-omit-mode`. +- Recursive copy (`C` on a directory), which v1 refuses. +- Auto-revert on external change — `pmacs.fs.watch` exists and polls + (`fs.lua:226`), so this is wiring plus a policy decision about polling a + directory the user is not looking at. +- Dired buffers in the desktop session — covered by Arc 3's standing + "non-file buffers in the desktop" deferral. +- Remote / Tramp-style paths. +- Symlink creation and symlink-target editing (the fixture rejects target + edits at commit; `init.lua:821-840`). + +## 14. Acceptance + +### Stage 0 — `find-file` (R2-6) + +0a. **Flat completion within the root.** With the current buffer in a + temp directory, `C-x C-f` offers that directory's entries as + candidates and does **not** offer entries of a subdirectory — + documenting the flat-source limitation as intended behavior rather + than leaving it unpinned. +0b. **Free-text accept of a deeper path.** Typing a full path below the + root and accepting opens that file, with no candidate selected + (`resolve_accepted_value`'s raw-input path). +0c. **Nonexistent path creates.** Accepting a path that does not exist + yields a buffer bound to it, unmodified and empty, with the + `[new file]` status — not an error. +0d. **No-path origin.** From a buffer with no backing path, the prompt + roots at the process cwd rather than erroring or offering nothing. + +### Stage 1 — the dired view + +1. **Listing shape.** `C-x d` on a temp directory renders a header line + plus one line per entry, with kind char, perms, size, mtime, and name; + a symlink renders `l` with ` -> target`; the entry count matches + `read_dir`. +2. **Visit dispatches on kind, through the panel-safe primitive + (Q#DR10).** `RET` on a subdirectory line opens that directory's dired + buffer; `RET` on a **file** line opens the file bound to its path (the + fixture's "not yet exposed" error is gone). `RET` on the header does + nothing. **The panel case is the real assertion**: with dired opened + `display = "panel"`, a `RET` on a file line leaves the dired panel + alive and puts the file in the document window — **falsified by + swapping `display_file` for `find_or_open`**, which must make the panel + disappear. +3. **Buffer-per-directory and canonicalization (Q#DR2).** Descending + twice then ascending twice yields the *same* buffer ids as the first + visit; every dired buffer's name matches the directory it displays; + and `C-x d` on `/tmp`, `/tmp/`, and `/tmp/../tmp` (with a real temp + dir) yields **one** buffer, not three. With + `dired.kill-when-opening = true`, the departed buffer is gone. +3b. **Canonicalization parity (R2-4).** One shared edge-case list — + `//tmp`, a trailing slash, `~` with `HOME` set and unset, `.`/`..` + segments including a `..` that would escape root, a relative path — + driven through **both** dired's canonicalizer and the Rust + `normalize_buffer_path`, asserting identical output. If Stage 1 + exposes the normalizer instead of mirroring it, this degenerates to a + round-trip test and the mirror-removal follow-up is dropped. +3c. **Panel descent (Q#DR10, R2-3).** With dired opened + `display = "panel"`, `RET` on a **directory** line leaves dired in the + same side window showing the new directory — the panel is neither + replaced by a document window nor duplicated — and a dedicated panel + is still dedicated afterward. +4. **Ownership check (Q#DR2, F7).** A foreign `pmacs.buffer.create` + buffer named exactly `*dired:*` is **not** adopted: `C-x d` on + that path leaves the foreign buffer's contents byte-identical and + opens dired elsewhere. +5. **Read-only (Q#DR3).** A `buffer.self-insert` into a dired buffer is + rejected by the intercept and leaves the text byte-identical; dired's + own repaint succeeds through `bypass_intercept`. `set_round_trip_input` + is set, pinned **through the real dispatch path** so a semantic + frontend's `d` reaches the binding rather than optimistic-applying — + falsified by reverting the `set_round_trip_input` call, not by a + direct-call assertion. +6. **Mode keymap (Q#DR8).** The keys resolve through `scope = "mode"` + with no per-buffer binding: a *second* dired buffer, created without + any `keymap.bind` call of its own, still responds to `g` and `^`. + `pmacs.buffer.major_mode(buf)` is `"dired"`, and the mode shows in the + statusline. +7. **Cursor preservation (Q#DR9).** With the cursor on entry `k`, `g` + re-seats on the same **basename** after an external file was added + *above* it (so the line index changed); when that basename is deleted + externally, the cursor lands on the nearest surviving line, not line 0. +8. **Sort.** `s` cycles name → mtime → size → name; mtime sorts newest + first and size largest first, each with a stable name tiebreak; the + cursor stays on its basename across the reorder. +9. **Tolerant listing (Q#DR6).** In a directory containing a child whose + `lstat` fails, `{ tolerant = true }` returns the surviving entries plus + one `errors` row naming the child; dired renders every readable entry + plus the unreadable-count footer; and **the default (non-opt) call + still returns a bare array** — both forms called in one test, so the + fixture's contract cannot regress unnoticed. A failure on the parent + directory itself is still fatal. +10. **Tolerant symlink targets (Q#DR6, F5).** A directory containing a + symlink whose target is non-UTF-8 lists successfully under + `{ tolerant = true }`, with that entry present and its target reported + unknown — **falsified by reverting the `read_link`/target arm**, which + must take the whole listing down. +11. **Unknown opts rejected (minor c).** `read_dir(path, { tolerat = true })` + errors naming the unknown key rather than silently listing in fatal + mode. +12. **Non-UTF-8 names stay fatal, and say so.** A directory containing a + non-UTF-8 *name* reports the structured `NonUtf8Path` error with the + offending bytes; dired surfaces it as a status message and creates no + buffer. +13. **`dired-jump`.** From a file buffer, `C-x C-j` opens dired on that + file's directory with the cursor on that file's line. From a buffer + with no path, it reports that and creates nothing. +14. **Quit.** `q` restores the previously active buffer; in a side window + (`display = "panel"`) it routes through `pmacs.window.quit`, matching + `listview.quit`'s Q#BP11b split. +15. **Failure leaves nothing behind.** `C-x d` on a nonexistent or + unreadable directory creates no buffer, switches no window, and + reports the reason — the fixture's + `dired_open_failure_leaves_editor_unchanged` invariant. +16. **Scale.** A 10,000-entry directory renders within the fixture's + established 200 ms budget, on the builtin path — carrying the same + `cfg_attr(target_os = "macos", ignore)` gate the fixture's version + uses (`m8_2_acceptance.rs:211-213`), since hosted macOS debug runners + do not consistently satisfy it. +17. **The fixture still passes.** `m8_2_acceptance` 15/15 and + `m8_3_acceptance` 32/32 unchanged, proving the `read_dir` opt is + additive. + +Every behavioral claim above is bite-verified with `scripts/bite`. + +## 15. Gates (Stage 1) + +`cargo fmt --check`; `cargo clippy --workspace --all-targets -- -D +warnings` as its own step; `cargo test --lib`; `cargo test --lib +--features crdt`; `tests/dired_acceptance.rs` (default + CRDT); +`tests/m8_1_acceptance.rs`, `tests/m8_2_acceptance.rs`, and +`tests/m8_3_acceptance.rs` (the additivity proof — m8_1 because it +exercises `pmacs.fs.rename` and `read_dir` directly); `cargo test --test +m4_acceptance -- --skip basedpyright`; `PMACS_REQUIRE_GPU=1 cargo test -p +pmacs-gpu`; the workspace sweep **with an isolated `XDG_CONFIG_HOME`** +(the real `~/.config/pmacs/init.lua` on this desktop calls +`install_local`, which races every editor the sweep builds and leaks a +status message into frame-comparing suites); `git diff --check`. + +## 16. Branch and PR plan + +Branch `dired`, worktree `../pmacs-dired-arc` — **not** `../pmacs-dired`, +which would read as the fixture. **Based on canonical `githubsucks/main` +@ `0827dd1`** (Lean 4 Stage 1 #160), which is one merge ahead of the +scout's `e745068`; see §2's base note for why that movement does not +disturb the ground truth. + +**The shared checkout is not the place to cut this.** It currently has +`lean4-stage1` checked out with in-progress foreign work +(`src/highlight.rs` modified), and this framing is untracked in it. Per +the §5 ops rule, the branch is cut as a **sibling worktree off `main`** +and the framing is committed there as the branch's first commit, rather +than by switching the shared checkout. The framing does not travel until +that commit is pushed. + +Stage 1 implements on the same branch and opens as the first dired PR. +Stages 2 and 3 are separate branches and PRs off the `main` that results +from the prior stage, each with its own detailed framing. If Stage 0 +(`find-file`) is taken separately it goes first, on its own branch +`find-file`, and Stage 1 rebases onto the resulting `main`. diff --git a/tests/find_file_acceptance.rs b/tests/find_file_acceptance.rs new file mode 100644 index 0000000..fb793e3 --- /dev/null +++ b/tests/find_file_acceptance.rs @@ -0,0 +1,373 @@ +// tests/find_file_acceptance.rs --- dired arc Stage 0 (`C-x C-f`) acceptance. + +//! Acceptance for `find-file`, the dired arc's Stage 0 +//! (`docs/dired-framing.md` §14, items 0a-0d, Q#DR11). +//! +//! Dispatch-driven throughout: the prompt is opened with a real +//! `C-x C-f`, filled by typing real keys, and completed with a real +//! RET. `pmacs.command.invoke` would bypass the binding (a dead +//! keymap entry would pass vacuously) and the Lua lifecycle +//! `minibuffer.accept()` bypasses the dispatch path interactive input +//! actually takes --- the editops suite's discipline, for the same +//! reasons. +//! +//! Fixtures use `.txt` files so no `buffer.after-load` hook spawns a +//! language server. + +use crossterm::event::{KeyCode, KeyEvent, KeyEventKind, KeyEventState, KeyModifiers}; +use pmacs::editor::EditorState; +use pmacs::protocol::FrontendId; + +fn key(code: KeyCode, mods: KeyModifiers) -> KeyEvent { + KeyEvent { + code, + modifiers: mods, + kind: KeyEventKind::Press, + state: KeyEventState::NONE, + } +} + +fn ctrl(s: &mut EditorState, c: char) { + s.dispatch_key( + FrontendId::LOCAL, + key(KeyCode::Char(c), KeyModifiers::CONTROL), + ); +} + +fn press(s: &mut EditorState, code: KeyCode) { + s.dispatch_key(FrontendId::LOCAL, key(code, KeyModifiers::NONE)); +} + +fn type_str(s: &mut EditorState, text: &str) { + for ch in text.chars() { + s.dispatch_key( + FrontendId::LOCAL, + key(KeyCode::Char(ch), KeyModifiers::NONE), + ); + } +} + +fn exec(s: &EditorState, src: &str) { + s.lua_host.lua().load(src.to_string()).exec().unwrap(); +} + +fn eval(s: &EditorState, src: &str) -> T { + s.lua_host.lua().load(src.to_string()).eval().unwrap() +} + +/// Open the find-file prompt through the real `C-x C-f` binding. +fn open_prompt(s: &mut EditorState) { + ctrl(s, 'x'); + ctrl(s, 'f'); + assert!( + eval::(s, "return pmacs.minibuffer.is_active()"), + "C-x C-f must open a minibuffer prompt" + ); +} + +/// The active buffer's backing path, or `None`. +fn active_path(s: &EditorState) -> Option { + eval::>( + s, + "local b = pmacs.window.buffer()\n\ + if b == nil then return nil end\n\ + local ok, p = pcall(function() return b:path() end)\n\ + if ok then return p end\n\ + return nil", + ) +} + +fn candidates(s: &EditorState) -> Vec { + eval::>(s, "return pmacs.minibuffer.candidates()") +} + +fn status(s: &EditorState) -> String { + s.core.borrow().status.clone() +} + +/// An editor whose active buffer is a real file inside `dir`, so +/// find-file's root resolves to that directory. +fn editor_in(dir: &std::path::Path) -> EditorState { + let anchor = dir.join("anchor.txt"); + std::fs::write(&anchor, b"anchor\n").expect("write anchor"); + let state = EditorState::new(); + state.lua_host.reopen_init_phase_for_testing(); + let anchor_str = anchor.display().to_string(); + exec( + &state, + &format!("pmacs.buffer.find_or_open({anchor_str:?})"), + ); + state +} + +/// 0a --- completion is flat: it offers the root's own entries and +/// never descends into a subdirectory. +#[test] +fn find_file_completion_lists_the_root_only_and_does_not_descend() { + let td = tempfile::tempdir().expect("tempdir"); + std::fs::write(td.path().join("alpha.txt"), b"a").expect("write"); + std::fs::create_dir(td.path().join("sub")).expect("mkdir"); + std::fs::write(td.path().join("sub").join("inner.txt"), b"i").expect("write"); + + let mut s = editor_in(td.path()); + open_prompt(&mut s); + + let cands = candidates(&s); + assert!( + cands.iter().any(|c| c == "alpha.txt"), + "root entry must be offered; got {cands:?}" + ); + assert!( + cands.iter().any(|c| c == "sub"), + "the subdirectory itself must be offered; got {cands:?}" + ); + assert!( + !cands.iter().any(|c| c == "inner.txt"), + "completion must NOT descend into subdirectories; got {cands:?}" + ); +} + +/// 0b --- free text carries the deeper case. `sub/inner.txt` matches no +/// bare-basename candidate, so it reaches `on_accept` verbatim and is +/// joined onto the prompt's root. +#[test] +fn find_file_free_text_opens_a_path_below_the_root() { + let td = tempfile::tempdir().expect("tempdir"); + std::fs::create_dir(td.path().join("sub")).expect("mkdir"); + let inner = td.path().join("sub").join("inner.txt"); + std::fs::write(&inner, b"deep contents\n").expect("write"); + + let mut s = editor_in(td.path()); + open_prompt(&mut s); + type_str(&mut s, "sub/inner.txt"); + + assert!( + candidates(&s).is_empty(), + "a needle containing '/' must filter every basename candidate away, \ + or the selection would shadow the typed text" + ); + + press(&mut s, KeyCode::Enter); + + let path = active_path(&s).expect("a file must be open"); + assert_eq!( + std::fs::canonicalize(&path).expect("canonicalize opened"), + std::fs::canonicalize(&inner).expect("canonicalize fixture"), + "free text must open the deeper path" + ); + let text: String = eval(&s, "return pmacs.window.buffer():slice(0, 13)"); + assert_eq!(text, "deep contents", "the file's real contents must load"); +} + +/// 0c --- a path that does not exist creates a `[new file]` buffer +/// bound to it, rather than erroring. The name contains a `/` so the +/// candidate list is empty and the typed text is what arrives (see +/// `find_file_selected_candidate_shadows_typed_text` for the other +/// half of that rule). +#[test] +fn find_file_nonexistent_path_creates_a_new_file_buffer() { + let td = tempfile::tempdir().expect("tempdir"); + std::fs::create_dir(td.path().join("sub")).expect("mkdir"); + let fresh = td.path().join("sub").join("brand-new.txt"); + assert!(!fresh.exists(), "fixture must not exist yet"); + + let mut s = editor_in(td.path()); + open_prompt(&mut s); + type_str(&mut s, "sub/brand-new.txt"); + press(&mut s, KeyCode::Enter); + + let path = active_path(&s).expect("a buffer must be bound to the new path"); + assert!( + path.ends_with("sub/brand-new.txt"), + "the buffer must be bound to the typed path; got {path}" + ); + let len: usize = eval(&s, "return pmacs.window.buffer():len()"); + assert_eq!(len, 0, "a new-file buffer starts empty"); + assert!( + !fresh.exists(), + "find-file must not create the file on disk --- only the buffer" + ); + let line = status(&s); + assert!( + line.contains("[new file]"), + "the new-file status must surface; got {line:?}" + ); +} + +/// The everyday new-file flow: a BARE name, no separator, matching no +/// existing entry. The candidate list empties on its own, so the typed +/// text arrives and joins onto the root. This is the path users hit +/// first, and it is the only route through `find_file_resolve` that +/// combines free text with a relative join. +#[test] +fn find_file_bare_new_name_creates_in_the_root() { + let td = tempfile::tempdir().expect("tempdir"); + let fresh = td.path().join("zzz.txt"); + + let mut s = editor_in(td.path()); + open_prompt(&mut s); + // "zzz.txt" is not a subsequence of "anchor.txt" (no 'z' in it), so + // nothing survives the filter and the typed name is what accepts. + type_str(&mut s, "zzz.txt"); + assert!( + candidates(&s).is_empty(), + "fixture premise: a bare non-matching name must empty the list; got {:?}", + candidates(&s) + ); + + press(&mut s, KeyCode::Enter); + + let path = active_path(&s).expect("a buffer must be bound to the new path"); + assert_eq!( + std::path::Path::new(&path).parent(), + Some(td.path()), + "a bare name must join onto the prompt's root; got {path}" + ); + assert!( + path.ends_with("zzz.txt"), + "the buffer must carry the typed name; got {path}" + ); + let len: usize = eval(&s, "return pmacs.window.buffer():len()"); + assert_eq!(len, 0, "a new-file buffer starts empty"); + assert!(!fresh.exists(), "nothing is written to disk until save"); +} + +/// The failure arm. Accepting a DIRECTORY candidate reaches +/// `display_file`, whose load fails (opening a directory succeeds, the +/// read does not), and the command's `pcall` must turn that into a +/// status message rather than letting the error escape mid-dispatch. +/// Without the `pcall` this test fails, which is the point --- the +/// guard is pinned through the real accept path, not asserted directly. +#[test] +fn find_file_accepting_a_directory_reports_instead_of_raising() { + let td = tempfile::tempdir().expect("tempdir"); + std::fs::create_dir(td.path().join("sub")).expect("mkdir"); + + let mut s = editor_in(td.path()); + let before = active_path(&s).expect("the anchor must be open"); + + open_prompt(&mut s); + // Only the directory matches: "anchor.txt" contains no 's'. + type_str(&mut s, "sub"); + assert_eq!( + candidates(&s), + vec!["sub".to_string()], + "fixture premise: the directory must be the sole candidate" + ); + + press(&mut s, KeyCode::Enter); + + let line = status(&s); + assert!( + line.starts_with("find-file: "), + "the failure must surface as this command's status message; got {line:?}" + ); + assert_eq!( + active_path(&s).as_deref(), + Some(before.as_str()), + "a failed open must leave the active buffer alone" + ); + assert!( + !eval::(&s, "return pmacs.minibuffer.is_active()"), + "the prompt must have closed even though the open failed" + ); +} + +/// 0d --- with no backing path, the prompt roots at the process cwd +/// (`source_root` is omitted, and the Rust side defaults to "."). +/// The test crate's cwd is the crate root, so `Cargo.toml` is a +/// stable, real candidate there. +#[test] +fn find_file_without_a_backing_path_roots_at_the_process_cwd() { + let mut s = EditorState::new(); + s.lua_host.reopen_init_phase_for_testing(); + assert!( + active_path(&s).is_none(), + "the scratch buffer must have no backing path" + ); + + open_prompt(&mut s); + + let cands = candidates(&s); + assert!( + cands.iter().any(|c| c == "Cargo.toml"), + "a pathless buffer must root the prompt at the process cwd; got {cands:?}" + ); + // The field must start EMPTY. Any prefill (e.g. Emacs's + // directory-in-the-field) would contain a `/`, which filters every + // basename candidate away and silently disables completion --- the + // reason the root is named in the prompt string instead. + let typed: String = eval(&s, "return pmacs.minibuffer.contents()"); + assert_eq!( + typed, "", + "the prompt field must start empty or completion is dead on arrival" + ); +} + +/// The documented hole in Q#DR11, pinned so it is a decision rather +/// than an accident: `recompute_candidates` selects index 0 whenever +/// the list is non-empty and `resolve_accepted_value` returns the +/// SELECTED CANDIDATE over the typed text, so typing a new bare name +/// that is a subsequence of an existing entry opens the existing file. +/// Fixing this needs a Rust change to accept semantics, which Stage 0 +/// deliberately does not make. +#[test] +fn find_file_selected_candidate_shadows_typed_text() { + let td = tempfile::tempdir().expect("tempdir"); + std::fs::write(td.path().join("notes.md"), b"existing\n").expect("write"); + + let mut s = editor_in(td.path()); + open_prompt(&mut s); + // "nots" is a subsequence of "notes.md", so the candidate survives + // the filter and shadows the typed name. + type_str(&mut s, "nots"); + assert_eq!( + candidates(&s), + vec!["notes.md".to_string()], + "the fixture depends on 'nots' matching 'notes.md'" + ); + + press(&mut s, KeyCode::Enter); + + let path = active_path(&s).expect("a file must be open"); + assert!( + path.ends_with("notes.md"), + "documented behavior: the selected candidate wins over typed text; got {path}" + ); +} + +/// A leading `~` is expanded before the path reaches the core. This +/// matters because `get_or_load_buffer` normalizes the path it STORES +/// but loads from the RAW one, so an unexpanded `~/...` would dedup +/// against an open buffer yet fail to load a file that is not open. +#[test] +fn find_file_expands_a_leading_tilde() { + let Some(home) = std::env::var_os("HOME") else { + eprintln!("HOME unset; skipping tilde expansion pin"); + return; + }; + let home = home.to_string_lossy().into_owned(); + if home.is_empty() || !std::path::Path::new(&home).is_dir() { + eprintln!("HOME is not a usable directory; skipping"); + return; + } + + let mut s = EditorState::new(); + s.lua_host.reopen_init_phase_for_testing(); + open_prompt(&mut s); + // Contains a '/', so the typed text reaches on_accept verbatim. + // The leaf does not exist, so this lands on the new-file path and + // touches no disk state. + type_str(&mut s, "~/pmacs-find-file-tilde-probe.txt"); + press(&mut s, KeyCode::Enter); + + let path = active_path(&s).expect("a buffer must be bound"); + assert!( + !path.contains('~'), + "the tilde must be expanded, not passed through; got {path}" + ); + assert!( + path.starts_with(&home), + "the expansion must use $HOME; got {path} with HOME={home}" + ); +}